[jira] [Commented] (KAFKA-5269) TransactionBounceTest occasionally fails due to partition errors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16018245#comment-16018245
 ] 

ASF GitHub Bot commented on KAFKA-5269:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/3094


> TransactionBounceTest occasionally fails due to partition errors
> 
>
> Key: KAFKA-5269
> URL: https://issues.apache.org/jira/browse/KAFKA-5269
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Apurva Mehta
>Assignee: Apurva Mehta
>Priority: Blocker
>  Labels: exactly-once
> Fix For: 0.11.0.0
>
>
> The test sometimes encounters a partition level error 
> `UNKNOWN_TOPIC_OR_PARTITION` for the output topic. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5269) TransactionBounceTest occasionally fails due to partition errors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016621#comment-16016621
 ] 

ASF GitHub Bot commented on KAFKA-5269:
---

GitHub user apurvam opened a pull request:

https://github.com/apache/kafka/pull/3094

KAFKA-5269: Correct handling of UNKNOWN_TOPIC_OR_PARTITION error 

We should retry AddPartitionsToTxnRequest and TxnOffsetCommitRequest when 
receiving an UNKNOWN_TOPIC_OR_PARTITION error.

As described in the JIRA: It turns out that the 
`UNKNOWN_TOPIC_OR_PARTITION` is returned from the request handler in KafkaAPis 
for the AddPartitionsToTxn and the TxnOffsetCommitRequest when the broker's 
metadata doesn't contain one or more partitions in the request. This can happen 
for instance when the broker is bounced and has not received the cluster 
metadata yet. 

We should retry in these cases, as this is the model followed by the 
consumer when committing offsets, and by the producer with a ProduceRequest.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/apurvam/kafka 
KAFKA-5269-handle-unknown-topic-partition-in-transaction-manager

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3094.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3094


commit da2e3af528540f73d6d0a35c4c51b8a8dc7eef0d
Author: Apurva Mehta 
Date:   2017-05-18T23:01:33Z

Retry AddPartitionsToTxnRequest and TxnOffsetCommitRequest when receiving 
an UNKNOWN_TOPIC_OR_PARTITION error.




> TransactionBounceTest occasionally fails due to partition errors
> 
>
> Key: KAFKA-5269
> URL: https://issues.apache.org/jira/browse/KAFKA-5269
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Apurva Mehta
>Assignee: Apurva Mehta
>Priority: Blocker
>  Labels: exactly-once
>
> The test sometimes encounters a partition level error 
> `UNKNOWN_TOPIC_OR_PARTITION` for the output topic. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5269) TransactionBounceTest occasionally fails due to partition errors

[Apurva Mehta (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16016496#comment-16016496
 ] 

Apurva Mehta commented on KAFKA-5269:
-

It turns out that the `UNKNOWN_TOPIC_OR_PARTITION` is returned from the request 
handler in KafkaAPis for the AddPartitionsToTxn and the TxnOffsetCommitRequest 
when the broker's metadata doesn't contain one or more partitions in the 
request. This can happen for instance when the broker is bounced and has not 
received the cluster metadata yet. 

The correct fix is simple: the client should retry the request when receiving 
this error. 

> TransactionBounceTest occasionally fails due to partition errors
> 
>
> Key: KAFKA-5269
> URL: https://issues.apache.org/jira/browse/KAFKA-5269
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Apurva Mehta
>Assignee: Apurva Mehta
>Priority: Blocker
>  Labels: exactly-once
>
> The test sometimes encounters a partition level error 
> `UNKNOWN_TOPIC_OR_PARTITION` for the output topic. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (KAFKA-5269) TransactionBounceTest occasionally fails due to partition errors

[Apurva Mehta (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014864#comment-16014864
 ] 

Apurva Mehta commented on KAFKA-5269:
-

A relevant stack trace:
{noformat}
[2017-05-17 15:15:38,145] ERROR aborting producer batches because the 
transaction manager is in an error state. 
(org.apache.kafka.clients.producer.internals.Sender:208)
org.apache.kafka.common.KafkaException: Unexpected error in 
TxnOffsetCommitResponse: This server does not host this topic-partition.
at 
org.apache.kafka.clients.producer.internals.TransactionManager$TxnOffsetCommitHandler.handleResponse(TransactionManager.java:855)
at 
org.apache.kafka.clients.producer.internals.TransactionManager$TxnRequestHandler.onComplete(TransactionManager.java:529)
at 
org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:100)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:378)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:196)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:152)
at java.lang.Thread.run(Thread.java:745)
[2017-05-17 15:15:38,150] ERR
{noformat}

> TransactionBounceTest occasionally fails due to partition errors
> 
>
> Key: KAFKA-5269
> URL: https://issues.apache.org/jira/browse/KAFKA-5269
> Project: Kafka
>  Issue Type: Sub-task
>Reporter: Apurva Mehta
>
> The test sometimes encounters a partition level error 
> `UNKNOWN_TOPIC_OR_PARTITION` for the output topic. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)