Re: [jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
It seems to me that in the most common case the partition isn't going to be magically created and in that case it's more helpful to quit with an error than to sit there waiting for the partition to be created.
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050917#comment-16050917 ] Paolo Patierno commented on KAFKA-5434: --- [~vahid] I got your point. Maybe we could have an option as --waiting-partition (or the opposite --check-partition-exists) in order to have both behaviours. Wdyt ? Hoping someone else jump into this. > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Paolo Patierno > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16050915#comment-16050915 ] Vahid Hashemian commented on KAFKA-5434: [~ppatierno] I look at this from a different angle. In the scenario you described the consumer is just waiting to receive messages in the specified partition. The partition may not exist yet, but it's always possible to expand the topic and add more partitions. At that point if a producer sends messages to that (now existing) partition the consumer would see them. So, in my opinion this is not a bug, because the consumer is just waiting (vs. hanging). I understand this is debatable and perhaps others can weigh in. > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Paolo Patierno > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048848#comment-16048848 ] ASF GitHub Bot commented on KAFKA-5434: --- GitHub user ppatierno opened a pull request: https://github.com/apache/kafka/pull/3335 KAFKA-5434: Console consumer hangs if not existing partition is specified Added checking partition exists before assign request You can merge this pull request into a Git repository by running: $ git pull https://github.com/ppatierno/kafka kafka-5434 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3335.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3335 commit 44bbb65994a957866ffc9b7574a4e40870c4f69f Author: ppatierno Date: 2017-06-14T08:22:53Z Added checking partition exists before assign request > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048054#comment-16048054 ] Paolo Patierno commented on KAFKA-5434: --- Thank you very much. I'll push a PR for that. > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048048#comment-16048048 ] Vahid Hashemian commented on KAFKA-5434: Yeah, I couldn't assign it to you either. You can take over the JIRA whenever you get access. > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048035#comment-16048035 ] Paolo Patierno commented on KAFKA-5434: --- [~vahid] I can't do that now :( I asked for being added to the contributors list but I'm not part of it yet. Thanks Paolo > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048030#comment-16048030 ] Vahid Hashemian commented on KAFKA-5434: [~ppatierno] Sure. Feel free to assign the JIRA to yourself. > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (KAFKA-5434) Console consumer hangs if not existing partition is specified
[ https://issues.apache.org/jira/browse/KAFKA-5434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047462#comment-16047462 ] Paolo Patierno commented on KAFKA-5434: --- Hi [~vahid] is it possible assigning to me this JIRA because I was already working on that for a proposal PR ? Thanks Paolo > Console consumer hangs if not existing partition is specified > - > > Key: KAFKA-5434 > URL: https://issues.apache.org/jira/browse/KAFKA-5434 > Project: Kafka > Issue Type: Bug > Components: tools >Reporter: Paolo Patierno >Assignee: Vahid Hashemian > > Hi, > if I specify the --partition option for the console consumer with a not > existing partition for a topic, the application hangs indefinitely. > Debugging the code I see that it asks for metadata but when it receives topic > information and it doesn't find the requested partition inside such metadata, > the code retries new time. > Could be it worst to check if the partition exists using the partitionFor > method before calling the assign in the seek of the BaseConsumer and throwing > an exception so printing an error on the console ? > Thanks, > Paolo -- This message was sent by Atlassian JIRA (v6.4.14#64029)
