[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15779104#comment-15779104 ] ASF GitHub Bot commented on KAFKA-2120: --- Github user pono closed the pull request at: https://github.com/apache/kafka/pull/1061 > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193917#comment-15193917 ] ASF GitHub Bot commented on KAFKA-2120: --- GitHub user christian-posta opened a pull request: https://github.com/apache/kafka/pull/1061 TRIVIAL: remove TODO in ConsumerNetworkClient after KAFKA-2120 You can merge this pull request into a Git repository by running: $ git pull https://github.com/christian-posta/kafka ceposta-trivial-remove-todo-after-KAFKA-2120 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1061.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1061 commit e5baf7cae7028c3840195328111fd45b4a403616 Author: Christian Posta Date: 2016-03-14T19:03:46Z TRIVIAL: remove TODO in ConsumerNetworkClient after KAFKA-2120 > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090458#comment-15090458 ] Nitin Padalia commented on KAFKA-2120: -- Is this resolution backported to older versions; i.e.; 0.8.2 also? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14960985#comment-14960985 ] ASF GitHub Bot commented on KAFKA-2120: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/320 > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959685#comment-14959685 ] ASF GitHub Bot commented on KAFKA-2120: --- GitHub user MayureshGharat opened a pull request: https://github.com/apache/kafka/pull/320 KAFKA-2120 Trivial fix to get rid of unused statements in kafkaProducer. You can merge this pull request into a Git repository by running: $ git pull https://github.com/MayureshGharat/kafka kafka-2120-followup Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/320.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #320 commit 2d28c58f79a7207729aaa92e0e255bbd7c566ba4 Author: Mayuresh Gharat Date: 2015-10-15T21:32:31Z Trivial fix to get rid of unused statements in kafkaProducer > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959196#comment-14959196 ] Mayuresh Gharat commented on KAFKA-2120: [~guozhang] Yes sure. Will submit a PR for it. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959179#comment-14959179 ] Guozhang Wang commented on KAFKA-2120: -- I see, that makes sense. Could we only define remainingTime in line 437 then and not getting return values for previous calls? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959152#comment-14959152 ] Mayuresh Gharat commented on KAFKA-2120: [~guozhang] The check is required to be done at each stage to see if we have crossed the maxBlockTime : 1) after fetching metadata. 2) after serialization of key. 3) after serialization of value. 4) after partitioning. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[
https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956184#comment-14956184
]
Guozhang Wang commented on KAFKA-2120:
--
[~mgharat] Your commit seems contains some debugging code in KafkaProducer like
{code}
remainingTime = checkMaybeGetRemainingTime(startTime);
{code}
Could you remove them?
> Add a request timeout to NetworkClient
> --
>
> Key: KAFKA-2120
> URL: https://issues.apache.org/jira/browse/KAFKA-2120
> Project: Kafka
> Issue Type: New Feature
>Reporter: Jiangjie Qin
>Assignee: Mayuresh Gharat
>Priority: Blocker
> Fix For: 0.9.0.0
>
> Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch,
> KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch,
> KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch,
> KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch,
> KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch,
> KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch,
> KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch
>
>
> Currently NetworkClient does not have a timeout setting for requests. So if
> no response is received for a request due to reasons such as broker is down,
> the request will never be completed.
> Request timeout will also be used as implicit timeout for some methods such
> as KafkaProducer.flush() and kafkaProducer.close().
> KIP-19 is created for this public interface change.
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944047#comment-14944047 ] ASF GitHub Bot commented on KAFKA-2120: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/278 > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944042#comment-14944042 ] ASF GitHub Bot commented on KAFKA-2120: --- GitHub user guozhangwang opened a pull request: https://github.com/apache/kafka/pull/278 TRIVIAL: remove TODO in KafkaConsumer after KAFKA-2120 You can merge this pull request into a Git repository by running: $ git pull https://github.com/guozhangwang/kafka TL-KafkaConsumer Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/278.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #278 commit 3c84b3d081962dfd29dccd4e053d05adca581ede Author: Guozhang Wang Date: 2015-10-05T21:04:08Z TRIVIAL: remove TODO in KafkaConsumer after KAFKA-2120 > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14934308#comment-14934308 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch, KAFKA-2120_2015-09-28_16:13:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14876838#comment-14876838 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch, > KAFKA-2120_2015-09-18_19:27:48.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14804570#comment-14804570 ] Joel Koshy commented on KAFKA-2120: --- Actually some of the confusion on my side is because I was running tests with JDK 7u51 which has been failing consistently for a while now (wonder what's up with that) - with JDK 8 the previous revision does seem to go through unit tests without failures. So in the interest of keeping trunk somewhat stable I have reverted the patch for now - sorry about the inconvenience. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803415#comment-14803415 ] Ismael Juma commented on KAFKA-2120: [~mgharat], it is hard to tell for sure given the number of tests that are sensitive to timing issues. The fact that the system tests are now failing does give credence to the fact that the latest changes have had some sort of effect though (although it could be a coincidence, of course): http://jenkins.confluent.io/job/kafka_system_tests/buildTimeTrend > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803338#comment-14803338 ] Mayuresh Gharat commented on KAFKA-2120: "Looking at the code, I noticed that we are passing the request timeout for the broker code (ControllerChannelManager, KafkaServer and `ReplicaFetcherThread`)". This might be true, I am not sure because anything that uses NetworkClient will use the new configured requestTimeout for sent requests. Also both me and Joel saw failures with earlier commits, I am not sure if they existed with earlier commits and this patch is making them more prominent. Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803312#comment-14803312 ] Ismael Juma commented on KAFKA-2120: Also, there are already failures in the system tests that I relaunched: http://jenkins.confluent.io/job/kafka_system_tests/77/console They had been successful for several days before this morning (they run once a day). > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803310#comment-14803310 ] Ismael Juma commented on KAFKA-2120: Tests pass for me when I run b658e25207174578c26ea94a75e5b0ea10f06ba5 (I know there are tests that have transient failures, but they happen rarely for me). With the latest code from trunk, I got at least one failure in 3 attempts with no successful run. This seems quite suspicious. Looking at the code, I noticed that we are are passing the request timeout for the broker code (ControllerChannelManager, KafkaServer and `ReplicaFetcherThread`). These classes were already handling the timeout themselves and I wonder if doing it in two places is causing some overlapping timeouts, which can delay things enough to cause test failures. Is it possible to just not set a request timeout in those cases? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803304#comment-14803304 ] Mayuresh Gharat commented on KAFKA-2120: [~ijuma] failures are consistent. I ran it multiple times. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803291#comment-14803291 ] Joel Koshy commented on KAFKA-2120: --- I see test failures on b7d4043d8da1d296519faf321e9f0f7188d7d67a as well > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803251#comment-14803251 ] Joel Koshy commented on KAFKA-2120: --- [~mgharat] yes I'm seeing the same thing. i.e., I ran tests against the revision just before your patch and I'm seeing various failures as well as hanging tests. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803252#comment-14803252 ] Ismael Juma commented on KAFKA-2120: Also, it doesn't look like there's any significant change between b7d4043d8da1d296519faf321e9f0f7188d7d67a (where tests passed) and the latest in trunk, or am I missing something? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803247#comment-14803247 ] Ismael Juma commented on KAFKA-2120: [~mgharat], are you getting consistent failures or are they transient? By the way, the ducktape system tests failed this morning, but were passing previously: http://jenkins.confluent.io/job/kafka_system_tests/ I started a new run in case the failure was transient. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14803215#comment-14803215 ] Mayuresh Gharat commented on KAFKA-2120: I am not sure if the failures are related to this patch. Here is what I did: 1) Checked out the lcommit : commit b658e25207174578c26ea94a75e5b0ea10f06ba5 Author: Parth Brahmbhatt Date: Mon Sep 14 15:02:32 2015 -0700 2) This is one commit before this patch. I ran ./gradlew test. 3) Got following errors, in different runs : *RUN 1 :* kafka.server.LogRecoveryTest > testHWCheckpointNoFailuresMultipleLogSegments FAILED java.lang.AssertionError: Failed to update high watermark for follower after timeout at org.junit.Assert.fail(Assert.java:88) at kafka.utils.TestUtils$.waitUntilTrue(TestUtils.scala:654) at kafka.server.LogRecoveryTest.testHWCheckpointNoFailuresMultipleLogSegments(LogRecoveryTest.scala:173) kafka.api.SSLConsumerTest > testGroupConsumption FAILED java.lang.IllegalStateException: Failed to consume the expected records after 301 iterations. at kafka.api.SSLConsumerTest.consumeRecords(SSLConsumerTest.scala:226) at kafka.api.SSLConsumerTest.testGroupConsumption(SSLConsumerTest.scala:165) kafka.integration.UncleanLeaderElectionTest > testCleanLeaderElectionDisabledByTopicOverride FAILED java.lang.AssertionError: expected: but was: at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:144) at kafka.integration.UncleanLeaderElectionTest.verifyUncleanLeaderElectionDisabled(UncleanLeaderElectionTest.scala:253) at kafka.integration.UncleanLeaderElectionTest.testCleanLeaderElectionDisabledByTopicOverride(UncleanLeaderElectionTest.scala:155) kafka.api.ConsumerTest > testPatternSubscription FAILED java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received. at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:56) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:43) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25) at kafka.api.ConsumerTest$$anonfun$sendRecords$1.apply(ConsumerTest.scala:449) at kafka.api.ConsumerTest$$anonfun$sendRecords$1.apply(ConsumerTest.scala:449) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) at scala.collection.IterableLike$class.foreach(IterableLike.scala:72) at scala.collection.AbstractIterable.foreach(Iterable.scala:54) at scala.collection.TraversableLike$class.map(TraversableLike.scala:244) at scala.collection.AbstractTraversable.map(Traversable.scala:105) at kafka.api.ConsumerTest.sendRecords(ConsumerTest.scala:449) at kafka.api.ConsumerTest.testPatternSubscription(ConsumerTest.scala:132) Caused by: org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received. kafka.api.ProducerBounceTest > testBrokerFailure FAILED java.lang.AssertionError at org.junit.Assert.fail(Assert.java:86) at org.junit.Assert.assertTrue(Assert.java:41) at org.junit.Assert.assertTrue(Assert.java:52) at kafka.api.ProducerBounceTest.testBrokerFailure(ProducerBounceTest.scala:117) kafka.api.ConsumerTest > testPatternSubscription FAILED java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received. at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:56) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:43) at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:25) at kafka.api.ConsumerTest$$anonfun$sendRecords$1.apply(ConsumerTest.scala:449) at kafka.api.ConsumerTest$$anonfun$sendRecords$1.apply(ConsumerTest.scala:449) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244) at scala.collection.Iterator$class.foreach(Iterator.scala:727) at scala.collection.AbstractIterator.forea
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791296#comment-14791296 ] Mayuresh Gharat commented on KAFKA-2120: Yep. Looking in to this. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791284#comment-14791284 ] Joel Koshy commented on KAFKA-2120: --- Hmm.. actually after rerunning tests I'm not sure this patch is not to blame. [~mgharat] can you also look into this. I'm going to do a few more runs with an earlier revision, but if it appears that this patch is making things worse I would be in favor of reverting if we cannot figure it out by EOD today. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14791049#comment-14791049 ] Joel Koshy commented on KAFKA-2120: --- Yes I did note failures in the RB, but those were on trunk as well and transient on my laptop. I believe the failures are Jenkins slowness, but I'm going to rerun tests locally to verify > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790914#comment-14790914 ] Ismael Juma commented on KAFKA-2120: The Jenkins builds failed with 11 failures, seems higher than normal, but could be a coincidence. Worth double-checking though. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790793#comment-14790793 ] Mayuresh Gharat commented on KAFKA-2120: Thanks a lot everyone who provided great reviews and insights. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1474#comment-1474 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.9.0.0 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch, KAFKA-2120_2015-09-15_18:57:20.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14741594#comment-14741594 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch, > KAFKA-2120_2015-09-11_14:54:15.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14740166#comment-14740166 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch, KAFKA-2120_2015-09-10_21:38:55.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739766#comment-14739766 ] Joel Koshy commented on KAFKA-2120: --- Thanks for the updated patch - I will review again later today > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738890#comment-14738890 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~junrao], Thanks for the comments. I have uploaded a new patch and also made some improvements as per [~jjkoshy]'s comments. Would you mind taking another look. Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737994#comment-14737994 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch, > KAFKA-2120_2015-09-09_18:56:18.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737819#comment-14737819 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch, KAFKA-2120_2015-09-09_16:45:44.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735899#comment-14735899 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~junrao], I have uploaded a new patch addressing the concerns that you raised and also explained my earlier approach. Thanks a lot for all the comments. Would you mind taking another look? Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731670#comment-14731670 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch, > KAFKA-2120_2015-09-04_17:49:01.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731570#comment-14731570 ] Tapasi Paul commented on KAFKA-2120: Hello folks, We are facing an issue with the networkproducerclient(rls 8.2.1) spinning issue as reported in this Jira , when kafka is down. Looks like there is a patch available and in the distribution I see rls 8.2.2 is now available. Can you please tell me if this fix for the timeout is there in 8.2.2 ? If not, when it would be available ? Thanks > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14729893#comment-14729893 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat >Priority: Blocker > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch, KAFKA-2120_2015-09-03_15:12:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14693951#comment-14693951 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch, > KAFKA-2120_2015-08-12_10:59:09.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14681145#comment-14681145 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch, KAFKA-2120_2015-08-10_19:55:18.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660405#comment-14660405 ] Jason Gustafson commented on KAFKA-2120: [~becket_qin] That is what I was thinking when I first looked at this patch, and it's still an option. However, since we are recommending session timeouts on the order of 60 seconds, it seems unfortunate to have to apply the same timeout on all requests (even those that should return immediately). Allowing a lower timeout for other requests means that the consumer could recover from worst-case failures much quicker. That being said, since the sanity check is easy, I think it's a good idea to add it to this patch. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660397#comment-14660397 ] Mayuresh Gharat commented on KAFKA-2120: [~becket_qin] yes we can do that. I will create a separate ticket and work on that. I think we should not include that in this patch if possible since this patch is specifically for KIP-19. Let us wait a bit and think on this a bit and then I will send out the jira links for the new tickets. Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660375#comment-14660375 ] Jiangjie Qin commented on KAFKA-2120: - [~hachikuji] [~mgharat], the request time is for exception handling purpose, the request timeout should be the highest timeout across all the timeout. Can we just sanity check the configurations user provides? The rules should be: Request timeout > session timeout. Request timeout > fetch.max.wait.timeout request timeout won't kick in before the other timeout reached. The sanity check patch should be relatively simple. We can either do it in this patch or in a separate patch. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660329#comment-14660329 ] Jason Gustafson commented on KAFKA-2120: [~mgharat] Sounds reasonable to me. Thanks! > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660314#comment-14660314 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~jasong35], Cool. So what we can do is : 1) Let this patch go through since this is important for the new producer to be used in production. 2) I will create separate ticket for #1 and #2 that you mentioned and we can work on that separately from this patch. Does that work? Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660303#comment-14660303 ] Jason Gustafson commented on KAFKA-2120: [~mgharat] Yeah, that's what I mean. I think the consumer has three different cases where a different timeout should be used: 1. Join group requests: The timeout should be based on the session timeout. 2. Fetch requests: The timeout should be based on the fetch max wait time. 3. All others: This includes heartbeats and metadata requests, and can use the default "request.timeout.ms" from this patch. So we don't actually need any new configuration, but it seems like we need to ability to set timeouts differently depending on the request type. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660276#comment-14660276 ] Mayuresh Gharat commented on KAFKA-2120: [~jasong35] I am not too familiar with the consumer code right now. From your comment above I get a feeling that the consumer and producer have different requirements. Also I see that consumer uses a ConsumerNetworkClient and we can have a separate timeout config for Consumer that can be passed to NetworkClient. By per-requests timeout, do you mean that different types of request can have different timeouts configured? If yes, it can be done. But then we might degrade the user experience wherein the user has to now configure different timeouts for different kinds of requests, which means that he has to have internal knowledge of different kinds of requests that clients are making to brokers. I might be wrong here since I am not fully aware of the consumer code base and I am still starting with it. Let me know your thoughts on this. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660242#comment-14660242 ] Glenn Sontheimer commented on KAFKA-2120: - Hello, I am out of the office and will return on August 10. Thanks. -Glenn > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660241#comment-14660241 ] Jason Gustafson commented on KAFKA-2120: [~mgharat] It occurred to me that the global request timeout doesn't work so well for the consumer. In particular, join group requests can take as long as the session timeout (which we encourage people to set high), but it would be unfortunate if we had to use that as the timeout for all requests. I wonder if it would be a good idea to have per-request timeouts? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Fix For: 0.8.3 > > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647423#comment-14647423 ] Stevo Slavic commented on KAFKA-2120: - If this is considered as must have for Kafka 0.8.3, please consider setting "Fix Version" to 0.8.3. Related issue, KAFKA-1788 is fixed only for 0.8.3. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14646887#comment-14646887 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch, > KAFKA-2120_2015-07-29_15:57:02.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643533#comment-14643533 ] Mayuresh Gharat commented on KAFKA-2120: Updated reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Attachments: KAFKA-2120.patch, KAFKA-2120_2015-07-27_15:31:19.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14643399#comment-14643399 ] Mayuresh Gharat commented on KAFKA-2120: Created reviewboard https://reviews.apache.org/r/36858/diff/ against branch origin/trunk > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > Attachments: KAFKA-2120.patch > > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596265#comment-14596265 ] Mayuresh Gharat commented on KAFKA-2120: Hi [~jasong35], we have began work on this from today. Will try to get this done asap. Thanks, Mayuresh > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14596242#comment-14596242 ] Jason Gustafson commented on KAFKA-2120: Hey [~mgharat] and [~becket_qin], how is this feature coming along? I'd be happy to help out if you don't have time for it. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573643#comment-14573643 ] Jason Gustafson commented on KAFKA-2120: The new consumer will also be able to take advantage of a request timeout in NetworkClient. Currently the consumer just blocks indefinitely on some requests, but we will probably need a timeout to resolve KAFKA-1894. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499210#comment-14499210 ] Suresh Srinivas commented on KAFKA-2120: [~ewencp], thanks for the explanation. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14499093#comment-14499093 ] Ewen Cheslack-Postava commented on KAFKA-2120: -- [~sureshms] They're related, but not the same thing. This JIRA is about the network requests made by the NetworkClient class, and is really about an internal implementation detail - we want to make sure the request times out so the client can make progress, even if it's never going to get a response. KAFKA-1788 is about timeouts for producer.send() calls which can never even make a network request because they can't make a connection to the broker. It's a user-facing issue. See the mailing list discussion thread linked in the KIP for more details. > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-2120) Add a request timeout to NetworkClient
[ https://issues.apache.org/jira/browse/KAFKA-2120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14498340#comment-14498340 ] Suresh Srinivas commented on KAFKA-2120: KAFKA-1788 which has been in the work for some time now addresses this, right? > Add a request timeout to NetworkClient > -- > > Key: KAFKA-2120 > URL: https://issues.apache.org/jira/browse/KAFKA-2120 > Project: Kafka > Issue Type: New Feature >Reporter: Jiangjie Qin >Assignee: Mayuresh Gharat > > Currently NetworkClient does not have a timeout setting for requests. So if > no response is received for a request due to reasons such as broker is down, > the request will never be completed. > Request timeout will also be used as implicit timeout for some methods such > as KafkaProducer.flush() and kafkaProducer.close(). > KIP-19 is created for this public interface change. > https://cwiki.apache.org/confluence/display/KAFKA/KIP-19+-+Add+a+request+timeout+to+NetworkClient -- This message was sent by Atlassian JIRA (v6.3.4#6332)
