[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch
[
https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992965#comment-16992965
]
ASF GitHub Bot commented on KAFKA-7698:
---
hachikuji commented on pull request #5992: KAFKA-7698: Kafka Broker fail to
start: ProducerFencedException throw…
URL: https://github.com/apache/kafka/pull/5992
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Broker fail to start: ProducerFencedException thrown from
> producerstatemanager.scala!checkProducerEpoch
> --
>
> Key: KAFKA-7698
> URL: https://issues.apache.org/jira/browse/KAFKA-7698
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Ming Liu
>Priority: Major
> Labels: easyfix
>
> During our operation of Kafka, we frequently saw this failure:
> There was an error in one of the threads during logs loading:
> org.apache.kafka.common.errors.ProducerFencedException:
> {code:java}
> [06:57:09,697] INFO [ProducerStateManager partition=interaction_events-127]
> Loading producer state from snapshot file
> '/data/disk5/kafka/interaction_events-127/092130764817.snapshot'
> (kafka.log.ProducerStateManager)
> [06:57:09,698] INFO [Log partition=interaction_events-127,
> dir=/data/disk5/kafka] Completed load of log with 11 segments, log start
> offset 91975003024 and log end offset 92130764817 in 12701 ms (kafka.log.Log)
> [06:57:09,701] ERROR There was an error in one of the threads during logs
> loading: org.apache.kafka.common.errors.ProducerFencedException: Producer's
> epoch is no longer valid. There is probably another producer with a newer
> epoch. 63 (request epoch), 66 (server epoch) (kafka.log.LogManager)
> [06:57:09,705] INFO [ProducerStateManager
> partition=client_interaction_events_authorid_enrichment-20] Writing producer
> snapshot at offset 92418754384 (kafka.log.ProducerStateManager)
> [06:57:09,707] ERROR [KafkaServer id=2] Fatal error during KafkaServer
> startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is
> no longer valid. There is probably another producer with a newer epoch. 63
> (request epoch), 66 (server epoch)
> {code:java}
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch
[
https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745676#comment-16745676
]
ASF GitHub Bot commented on KAFKA-7698:
---
mingaliu commented on pull request #5992: KAFKA-7698: Kafka Broker fail to
start: ProducerFencedException throw…
URL: https://github.com/apache/kafka/pull/5992
If ValidationType is None, also skip the check in appendEndTxnMarker
(similar to append).
Verified with existing unitest and our daily operation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Broker fail to start: ProducerFencedException thrown from
> producerstatemanager.scala!checkProducerEpoch
> --
>
> Key: KAFKA-7698
> URL: https://issues.apache.org/jira/browse/KAFKA-7698
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Ming Liu
>Priority: Major
> Labels: easyfix
>
> During our operation of Kafka, we frequently saw this failure:
> There was an error in one of the threads during logs loading:
> org.apache.kafka.common.errors.ProducerFencedException:
> {code:java}
> [06:57:09,697] INFO [ProducerStateManager partition=interaction_events-127]
> Loading producer state from snapshot file
> '/data/disk5/kafka/interaction_events-127/092130764817.snapshot'
> (kafka.log.ProducerStateManager)
> [06:57:09,698] INFO [Log partition=interaction_events-127,
> dir=/data/disk5/kafka] Completed load of log with 11 segments, log start
> offset 91975003024 and log end offset 92130764817 in 12701 ms (kafka.log.Log)
> [06:57:09,701] ERROR There was an error in one of the threads during logs
> loading: org.apache.kafka.common.errors.ProducerFencedException: Producer's
> epoch is no longer valid. There is probably another producer with a newer
> epoch. 63 (request epoch), 66 (server epoch) (kafka.log.LogManager)
> [06:57:09,705] INFO [ProducerStateManager
> partition=client_interaction_events_authorid_enrichment-20] Writing producer
> snapshot at offset 92418754384 (kafka.log.ProducerStateManager)
> [06:57:09,707] ERROR [KafkaServer id=2] Fatal error during KafkaServer
> startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is
> no longer valid. There is probably another producer with a newer epoch. 63
> (request epoch), 66 (server epoch)
> {code:java}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch
[
https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16745675#comment-16745675
]
ASF GitHub Bot commented on KAFKA-7698:
---
mingaliu commented on pull request #5992: KAFKA-7698: Kafka Broker fail to
start: ProducerFencedException throw…
URL: https://github.com/apache/kafka/pull/5992
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Broker fail to start: ProducerFencedException thrown from
> producerstatemanager.scala!checkProducerEpoch
> --
>
> Key: KAFKA-7698
> URL: https://issues.apache.org/jira/browse/KAFKA-7698
> Project: Kafka
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Ming Liu
>Priority: Major
> Labels: easyfix
>
> During our operation of Kafka, we frequently saw this failure:
> There was an error in one of the threads during logs loading:
> org.apache.kafka.common.errors.ProducerFencedException:
> {code:java}
> [06:57:09,697] INFO [ProducerStateManager partition=interaction_events-127]
> Loading producer state from snapshot file
> '/data/disk5/kafka/interaction_events-127/092130764817.snapshot'
> (kafka.log.ProducerStateManager)
> [06:57:09,698] INFO [Log partition=interaction_events-127,
> dir=/data/disk5/kafka] Completed load of log with 11 segments, log start
> offset 91975003024 and log end offset 92130764817 in 12701 ms (kafka.log.Log)
> [06:57:09,701] ERROR There was an error in one of the threads during logs
> loading: org.apache.kafka.common.errors.ProducerFencedException: Producer's
> epoch is no longer valid. There is probably another producer with a newer
> epoch. 63 (request epoch), 66 (server epoch) (kafka.log.LogManager)
> [06:57:09,705] INFO [ProducerStateManager
> partition=client_interaction_events_authorid_enrichment-20] Writing producer
> snapshot at offset 92418754384 (kafka.log.ProducerStateManager)
> [06:57:09,707] ERROR [KafkaServer id=2] Fatal error during KafkaServer
> startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is
> no longer valid. There is probably another producer with a newer epoch. 63
> (request epoch), 66 (server epoch)
> {code:java}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch
[
https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707628#comment-16707628
]
ASF GitHub Bot commented on KAFKA-7698:
---
mingaliu opened a new pull request #5992: KAFKA-7698: Kafka Broker fail to
start: ProducerFencedException throw…
URL: https://github.com/apache/kafka/pull/5992
If ValidationType is None, also skip the check in appendEndTxnMarker
(similar to append).
Verified with existing unitest and our daily operation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Kafka Broker fail to start: ProducerFencedException thrown from
> producerstatemanager.scala!checkProducerEpoch
> --
>
> Key: KAFKA-7698
> URL: https://issues.apache.org/jira/browse/KAFKA-7698
> Project: Kafka
> Issue Type: Bug
>Reporter: Ming Liu
>Priority: Major
>
> During our operation of Kafka, we frequently saw this failure:
> There was an error in one of the threads during logs loading:
> org.apache.kafka.common.errors.ProducerFencedException:
> {code:java}
> [06:57:09,697] INFO [ProducerStateManager partition=interaction_events-127]
> Loading producer state from snapshot file
> '/data/disk5/kafka/interaction_events-127/092130764817.snapshot'
> (kafka.log.ProducerStateManager)
> [06:57:09,698] INFO [Log partition=interaction_events-127,
> dir=/data/disk5/kafka] Completed load of log with 11 segments, log start
> offset 91975003024 and log end offset 92130764817 in 12701 ms (kafka.log.Log)
> [06:57:09,701] ERROR There was an error in one of the threads during logs
> loading: org.apache.kafka.common.errors.ProducerFencedException: Producer's
> epoch is no longer valid. There is probably another producer with a newer
> epoch. 63 (request epoch), 66 (server epoch) (kafka.log.LogManager)
> [06:57:09,705] INFO [ProducerStateManager
> partition=client_interaction_events_authorid_enrichment-20] Writing producer
> snapshot at offset 92418754384 (kafka.log.ProducerStateManager)
> [06:57:09,707] ERROR [KafkaServer id=2] Fatal error during KafkaServer
> startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is
> no longer valid. There is probably another producer with a newer epoch. 63
> (request epoch), 66 (server epoch)
> {code:java}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-7698) Kafka Broker fail to start: ProducerFencedException thrown from producerstatemanager.scala!checkProducerEpoch
[
https://issues.apache.org/jira/browse/KAFKA-7698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16707621#comment-16707621
]
Ming Liu commented on KAFKA-7698:
-
The problem is seems coming from appendEndTxnMarker of
ProducerStateManager.scala.
ProducerAppendInfo() has this ValidationType passed in. During initial
bootstrapping, ValidationType is set to None (instead of Client or All).
So, for Append(), maybeValidateAppend() is called and the check is skipped
during loading phase.
But for appendEndTxnMarker(), it seems we should also skip the check.
> Kafka Broker fail to start: ProducerFencedException thrown from
> producerstatemanager.scala!checkProducerEpoch
> --
>
> Key: KAFKA-7698
> URL: https://issues.apache.org/jira/browse/KAFKA-7698
> Project: Kafka
> Issue Type: Bug
>Reporter: Ming Liu
>Priority: Major
>
> During our operation of Kafka, we frequently saw this failure:
> There was an error in one of the threads during logs loading:
> org.apache.kafka.common.errors.ProducerFencedException:
> {code:java}
> [06:57:09,697] INFO [ProducerStateManager partition=interaction_events-127]
> Loading producer state from snapshot file
> '/data/disk5/kafka/interaction_events-127/092130764817.snapshot'
> (kafka.log.ProducerStateManager)
> [06:57:09,698] INFO [Log partition=interaction_events-127,
> dir=/data/disk5/kafka] Completed load of log with 11 segments, log start
> offset 91975003024 and log end offset 92130764817 in 12701 ms (kafka.log.Log)
> [06:57:09,701] ERROR There was an error in one of the threads during logs
> loading: org.apache.kafka.common.errors.ProducerFencedException: Producer's
> epoch is no longer valid. There is probably another producer with a newer
> epoch. 63 (request epoch), 66 (server epoch) (kafka.log.LogManager)
> [06:57:09,705] INFO [ProducerStateManager
> partition=client_interaction_events_authorid_enrichment-20] Writing producer
> snapshot at offset 92418754384 (kafka.log.ProducerStateManager)
> [06:57:09,707] ERROR [KafkaServer id=2] Fatal error during KafkaServer
> startup. Prepare to shutdown (kafka.server.KafkaServer)
> org.apache.kafka.common.errors.ProducerFencedException: Producer's epoch is
> no longer valid. There is probably another producer with a newer epoch. 63
> (request epoch), 66 (server epoch)
> {code:java}
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
