[ https://issues.apache.org/jira/browse/KAFKA-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Guozhang Wang resolved KAFKA-6212. ---------------------------------- Resolution: Cannot Reproduce > Kafka Streams - Incorrect partition rebalancing > ----------------------------------------------- > > Key: KAFKA-6212 > URL: https://issues.apache.org/jira/browse/KAFKA-6212 > Project: Kafka > Issue Type: Bug > Components: streams > Affects Versions: 0.10.0.1 > Environment: Linux > Reporter: Ivan Atanasov > Priority: Major > > Trying to use streaming with version 0.10.0.1 of kafka but it is not working > how I'd expect. I realize that this is a fairly old version now but it is > what we are running and are not in a position to upgrade right now. > The particular problem I am having is when an extra instance of the streaming > app is run using the same application ID. What seems to happen is the newly > introduced instance takes half of the partitions available, which is expected > but the original instance drops all the partitions it was reading from. > therefore from then on, data is only read from half the partitions. > Strangely offsets are still being committed for the other partitions but the > data from them is not consumed as expected. > My topology is very simple for now, all it does is a print of the message. > Also I have tried making both instances use different client IDs and state > directories. > Is this a known bug in 0.10.0.1? > *Logs Below:* > Instance 1: > {quote}[2017-11-15 10:41:41,597] INFO [StreamThread-1] Setting newly assigned > partitions [rawEvents-5, rawEvents-6, rawEvents-3, rawEvents-4, rawEvents-9, > rawEvents-7, rawEvents-8, rawEvents-1, rawEvents-2, rawEvents-0] for group > kafka-stream-test > (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) > [2017-11-15 10:41:41,616] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_0 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,645] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_1 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,645] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_2 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_3 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_4 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,646] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_5 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_6 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_7 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,647] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_8 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:41:41,648] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_9 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:42:08,682] INFO [StreamThread-1] Revoking previously assigned > partitions [rawEvents-5, rawEvents-6, rawEvents-3, rawEvents-4, rawEvents-9, > rawEvents-7, rawEvents-8, rawEvents-1, rawEvents-2, rawEvents-0] for group > kafka-stream-test > (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) > [2017-11-15 10:42:08,682] INFO [StreamThread-1] Removing a task 0_0 > (org.apache.kafka.streams.processor.internals.StreamThread){quote} > Instance 2: > {quote}[2017-11-15 10:42:08,827] INFO [StreamThread-1] Successfully joined > group kafka-stream-test with generation 2 > (org.apache.kafka.clients.consumer.internals.AbstractCoordinator) > [2017-11-15 10:42:08,829] INFO [StreamThread-1] Setting newly assigned > partitions [rawEvents-5, rawEvents-3, rawEvents-1, rawEvents-2, rawEvents-0] > for group kafka-stream-test > (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator) > [2017-11-15 10:42:08,840] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_0 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:42:08,869] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_1 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:42:08,870] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_2 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:42:08,871] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_3 > (org.apache.kafka.streams.processor.internals.StreamTask) > [2017-11-15 10:42:08,872] INFO [StreamThread-1] Creating restoration consumer > client for stream task #0_5 > (org.apache.kafka.streams.processor.internals.StreamTask){quote} -- This message was sent by Atlassian Jira (v8.3.4#803005)