[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16831738#comment-16831738
]
ASF GitHub Bot commented on KAFKA-8248:
---
hachikuji commented on pull request #6613: KAFKA-8248; Ensure time updated
before sending transactional request
URL: https://github.com/apache/kafka/pull/6613
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Assignee: Jason Gustafson
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16822391#comment-16822391
]
ASF GitHub Bot commented on KAFKA-8248:
---
hachikuji commented on pull request #6613: KAFKA-8248; Ensure time updated
before sending transactional request
URL: https://github.com/apache/kafka/pull/6613
This patch fixes a bug in the sending of transactional requests. We need to
call `KafkaClient.send` with an updated current time. Failing to do so can
result in an `IllegalStateExcepton` which leaves the producer effectively dead
since the in-flight correlation id has been set, but no request has been sent.
To avoid the same problem in the future, we update the in flight correlationId
only after sending the request.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821561#comment-16821561
]
ASF GitHub Bot commented on KAFKA-8248:
---
mjsax commented on pull request #6607: KAFKA-8248: Fix IllegalStateException in
Producer
URL: https://github.com/apache/kafka/pull/6607
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16821393#comment-16821393
]
Matthias J. Sax commented on KAFKA-8248:
It was reported for `2.0.0` but we assume it affects newer (and even older
versions), too.
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Assignee: Matthias J. Sax
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820743#comment-16820743
]
Ayusman Dikshit commented on KAFKA-8248:
Does this issue ONLY apply to 2.0.0 or later versions as well?
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8248) Producer may fail IllegalStateException
[
https://issues.apache.org/jira/browse/KAFKA-8248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16820597#comment-16820597
]
Jason Gustafson commented on KAFKA-8248:
Looking at the code, I think the problem might be that we don't update the time
before calling `client.send()` inside `maybeSendTransactionalRequest`.
Basically `NetworkClient.isReady` might return a different value at different
times, so if the latest time is not used, it might return incorrectly.
> Producer may fail IllegalStateException
> ---
>
> Key: KAFKA-8248
> URL: https://issues.apache.org/jira/browse/KAFKA-8248
> Project: Kafka
> Issue Type: Bug
> Components: producer
>Affects Versions: 2.0.0
>Reporter: Matthias J. Sax
>Priority: Major
>
> In a Kafka Streams application, we observed the following log from the
> producer:
> {quote}2019-04-17T01:58:25.898Z 17466081 [kafka-producer-network-thread |
> client-id-enrichment-gcoint-StreamThread-7-0_10-producer] ERROR
> org.apache.kafka.clients.producer.internals.Sender - [Producer
> clientId=client-id-enrichment-gcoint-StreamThread-7-0_10-producer,
> transactionalId=application-id-enrichment-gcoint-0_10] Uncaught error in
> kafka producer I/O thread:
> 2019-04-17T01:58:25.898Z java.lang.IllegalStateException: Attempt to send a
> request to node 1 which is not ready.
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.doSend(NetworkClient.java:430)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.NetworkClient.send(NetworkClient.java:411)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.maybeSendTransactionalRequest(Sender.java:362)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:214)
> 2019-04-17T01:58:25.898Z at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
> 2019-04-17T01:58:25.898Z at java.lang.Thread.run(Thread.java:748)
> {quote}
> Later, Kafka Streams (running with EOS enabled) shuts down with a
> `TimeoutException` that occurs during rebalance. It seem that the above error
> results in this `TimeoutException`. However, and `IllegalStateException` seem
> to indicate a bug in the producer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
