[
https://issues.apache.org/jira/browse/KAFKA-18018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Abhinav Dixit resolved KAFKA-18018.
-----------------------------------
Resolution: Later
> Consumption degrading when using DefaultStatePersister for 1 million records
> of size 1KB
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-18018
> URL: https://issues.apache.org/jira/browse/KAFKA-18018
> Project: Kafka
> Issue Type: Sub-task
> Reporter: Abhinav Dixit
> Assignee: Abhinav Dixit
> Priority: Major
>
> I was running some performance tests and I couldn't consume 1,000,000 records
> using 10 share consumers. I've narrowed down the issue to
> {{DefaultStatePersister}} . When I run the same tests using
> {{{}NoOpShareStatePersister{}}}, then I don't see the reliability issue.
> Steps to reproduce (could be maybe reproduced by other ways as well, but this
> is how I noticed it consistently) - # Create a topic with a single partition
> # Produce 1000 records into the topic and consume it with the help of 10
> share consumers.
> # Again produce 1000 records into the topic and consume it with the help of
> 10 share consumers.
> # Produce 1,000,000 records into the topic and consume it with the help of
> 10 share consumers.
> You'll see the records not getting consumed even within 5 minutes (It
> shouldn't take more than 5-6 seconds to consume 1,000,000 records with 10
> share consumers, based on my past experience)
> PS - I haven't noticed any issues using console share consumers, so probably
> an issue with scale
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
