adixitconfluent commented on PR #18864: URL: https://github.com/apache/kafka/pull/18864#issuecomment-2678917068
> > > can you please share the numbers with 16 partitions and 25 share consumers, with and without the PR. > > > > > > @apoorvmittal10 , here are the numbers for 16 partitions and 25 share consumers - > > With PR - 1 million records of size 1024 each - 17.5 seconds 5 million records of size 1024 each - 91 seconds > > Without PR - 1 million records of size 1024 each - 14.1 seconds 5 million records of size 1024 each - 72.2 seconds > > As mentioned above as well, this PR reduces the sharing of topic partitions from the assignor, hence the decline in performance is expected. With the future PRs, the performance should reach an optimum number. > > Just to clarify, how was the partition allocation with the current PR code. Also if members are removed and added then there would be more sharing of partitions as per the combine logic in the PR, correct? Will it affect the performance? right now, most of the members had 1-2 topic partitions allocated to them excepted 1-2 members which had a good 12-14 partitions assigned to them. Yes, if members are removed and added then there would be more sharing of partitions as per the combine logic in the PR. Given the small size of 1-5 million records, it should improve the performance. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
