[jira] [Resolved] (KAFKA-10513) Newly added topic or partitions are not assigned to running consumer groups using static membership

2020-10-02 Thread Marlon Ou (Jira) 


 [ 
https://issues.apache.org/jira/browse/KAFKA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marlon Ou resolved KAFKA-10513.
---
Resolution: Won't Fix

> Newly added topic or partitions are not assigned to running consumer groups 
> using static membership
> ---
>
> Key: KAFKA-10513
> URL: https://issues.apache.org/jira/browse/KAFKA-10513
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Affects Versions: 2.6.0
>Reporter: Marlon Ou
>Priority: Major
>
> If consumers are polling messages from a certain topic with static membership 
> and we add new partitions to this topic while the consumers are running, no 
> partition reassignment is ever triggered (and hence messages published into 
> the new partitions are never consumed). 
> To reproduce, simply set group instance IDs on the consumers: 
> {code:java}
> props.setProperty("group.instance.id", instanceId);
> {code}
> And then while the static consumers are running, use Kafka's admin client to 
> add more partitions to the topic:
> {code:java}
> adminClient.createPartitions(...)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (KAFKA-10513) Newly added topic or partitions are not assigned to running consumer groups using static membership

2020-09-29 Thread Marlon Ou (Jira) 


[ 
https://issues.apache.org/jira/browse/KAFKA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17204432#comment-17204432
 ] 

Marlon Ou commented on KAFKA-10513:
---

Thank you both so much for the help! I tried setting metadata.max.age.ms to a 
small value and it did work. The reason why I didn't see rebalance before is 
that the default value for metadata.max.age.ms is 5 minutes and I never waited 
past 5 mins for rebalancing. 

> Newly added topic or partitions are not assigned to running consumer groups 
> using static membership
> ---
>
> Key: KAFKA-10513
> URL: https://issues.apache.org/jira/browse/KAFKA-10513
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Affects Versions: 2.6.0
>Reporter: Marlon Ou
>Priority: Major
>
> If consumers are polling messages from a certain topic with static membership 
> and we add new partitions to this topic while the consumers are running, no 
> partition reassignment is ever triggered (and hence messages published into 
> the new partitions are never consumed). 
> To reproduce, simply set group instance IDs on the consumers: 
> {code:java}
> props.setProperty("group.instance.id", instanceId);
> {code}
> And then while the static consumers are running, use Kafka's admin client to 
> add more partitions to the topic:
> {code:java}
> adminClient.createPartitions(...)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (KAFKA-10513) Newly added topic or partitions are not assigned to running consumer groups using static membership

2020-09-24 Thread Marlon Ou (Jira) 


 [ 
https://issues.apache.org/jira/browse/KAFKA-10513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marlon Ou updated KAFKA-10513:
--
Summary: Newly added topic or partitions are not assigned to running 
consumer groups using static membership  (was: Newly added partitions are not 
assigned to running static consumer)

> Newly added topic or partitions are not assigned to running consumer groups 
> using static membership
> ---
>
> Key: KAFKA-10513
> URL: https://issues.apache.org/jira/browse/KAFKA-10513
> Project: Kafka
>  Issue Type: Bug
>  Components: consumer
>Affects Versions: 2.6.0
>Reporter: Marlon Ou
>Priority: Major
>
> If consumers are polling messages from a certain topic with static membership 
> and we add new partitions to this topic while the consumers are running, no 
> partition reassignment is ever triggered (and hence messages published into 
> the new partitions are never consumed). 
> To reproduce, simply set group instance IDs on the consumers: 
> {code:java}
> props.setProperty("group.instance.id", instanceId);
> {code}
> And then while the static consumers are running, use Kafka's admin client to 
> add more partitions to the topic:
> {code:java}
> adminClient.createPartitions(...)
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (KAFKA-10513) Newly added partitions are not assigned to running static consumer

2020-09-22 Thread Marlon Ou (Jira) 
Marlon Ou created KAFKA-10513:
-

 Summary: Newly added partitions are not assigned to running static 
consumer
 Key: KAFKA-10513
 URL: https://issues.apache.org/jira/browse/KAFKA-10513
 Project: Kafka
  Issue Type: Bug
  Components: consumer
Affects Versions: 2.6.0
Reporter: Marlon Ou


If consumers are polling messages from a certain topic with static membership 
and we add new partitions to this topic while the consumers are running, no 
partition reassignment is ever triggered (and hence messages published into the 
new partitions are never consumed). 

To reproduce, simply set group instance IDs on the consumers: 
{code:java}
props.setProperty("group.instance.id", instanceId);
{code}
And then while the static consumers are running, use Kafka's admin client to 
add more partitions to the topic:
{code:java}
adminClient.createPartitions(...)
{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)