[jira] [Commented] (KAFKA-2792) KafkaConsumer.close() can block unnecessarily due to leave group waiting for a reply

2015-11-10 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999069#comment-14999069
 ] 

ASF GitHub Bot commented on KAFKA-2792:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/480


> KafkaConsumer.close() can block unnecessarily due to leave group waiting for 
> a reply
> 
>
> Key: KAFKA-2792
> URL: https://issues.apache.org/jira/browse/KAFKA-2792
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Reporter: Ewen Cheslack-Postava
>Assignee: Ewen Cheslack-Postava
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The current implementation of close() waits for a response to LeaveGroup. 
> However, if we have an outstanding rebalance in the works, this can cause the 
> close() operation to have to wait for the entire rebalance process to 
> complete, which is annoying since the goal is to get rid of the consumer 
> object anyway. This is at best surprising and at worst can cause unexpected 
> bugs due to close() taking excessively long -- this was found due to 
> exceeding timeouts unexpectedly causing other operations in Kafka Connect to 
> timeout.
> Waiting for a response isn't necessary since as soon as the data is in the 
> TCP buffer, it'll be delivered to the broker. The client doesn't benefit at 
> all from seeing the close group. So we can instead just always send the 
> request 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2792) KafkaConsumer.close() can block unnecessarily due to leave group waiting for a reply

2015-11-10 Thread Guozhang Wang (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14999211#comment-14999211
 ] 

Guozhang Wang commented on KAFKA-2792:
--

[~ewencp] [~onurkaraman] If we fire-and-forget in the close() call, it will 
likely lead to an EOF exception on the server side whenever some consumer 
closes themselves, causing some log pollutions. This is why I was deciding to 
not do fire-and-forget in close() but only in unsubscribe(). We need to think 
about if this issue can be resolved in the socket server if we stick to this 
plan.

> KafkaConsumer.close() can block unnecessarily due to leave group waiting for 
> a reply
> 
>
> Key: KAFKA-2792
> URL: https://issues.apache.org/jira/browse/KAFKA-2792
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Reporter: Ewen Cheslack-Postava
>Assignee: Ewen Cheslack-Postava
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The current implementation of close() waits for a response to LeaveGroup. 
> However, if we have an outstanding rebalance in the works, this can cause the 
> close() operation to have to wait for the entire rebalance process to 
> complete, which is annoying since the goal is to get rid of the consumer 
> object anyway. This is at best surprising and at worst can cause unexpected 
> bugs due to close() taking excessively long -- this was found due to 
> exceeding timeouts unexpectedly causing other operations in Kafka Connect to 
> timeout.
> Waiting for a response isn't necessary since as soon as the data is in the 
> TCP buffer, it'll be delivered to the broker. The client doesn't benefit at 
> all from seeing the close group. So we can instead just always send the 
> request 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (KAFKA-2792) KafkaConsumer.close() can block unnecessarily due to leave group waiting for a reply

2015-11-09 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/KAFKA-2792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14997904#comment-14997904
 ] 

ASF GitHub Bot commented on KAFKA-2792:
---

GitHub user ewencp opened a pull request:

https://github.com/apache/kafka/pull/480

KAFKA-2792: Don't wait for a response to the leave group message when 
closing the new consumer.



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ewencp/kafka 
kafka-2792-fix-blocking-consumer-close

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/480.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #480


commit a781fba82af6a6224f441885e22d3d6b35fc96f0
Author: Ewen Cheslack-Postava 
Date:   2015-11-10T02:56:08Z

KAFKA-2792: Don't wait for a response to the leave group message when 
closing the new consumer.




> KafkaConsumer.close() can block unnecessarily due to leave group waiting for 
> a reply
> 
>
> Key: KAFKA-2792
> URL: https://issues.apache.org/jira/browse/KAFKA-2792
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Reporter: Ewen Cheslack-Postava
>Assignee: Ewen Cheslack-Postava
>Priority: Blocker
> Fix For: 0.9.0.0
>
>
> The current implementation of close() waits for a response to LeaveGroup. 
> However, if we have an outstanding rebalance in the works, this can cause the 
> close() operation to have to wait for the entire rebalance process to 
> complete, which is annoying since the goal is to get rid of the consumer 
> object anyway. This is at best surprising and at worst can cause unexpected 
> bugs due to close() taking excessively long -- this was found due to 
> exceeding timeouts unexpectedly causing other operations in Kafka Connect to 
> timeout.
> Waiting for a response isn't necessary since as soon as the data is in the 
> TCP buffer, it'll be delivered to the broker. The client doesn't benefit at 
> all from seeing the close group. So we can instead just always send the 
> request 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)