RivenSun2 edited a comment on pull request #11340: URL: https://github.com/apache/kafka/pull/11340#issuecomment-942858528
@guozhangwang Thank you for your reply 1. I agree that deleting the poll(long) method will make the code changes easier, but this cannot solve the root cause, `time.timer(rebalanceTimeoutMs)` is applied to the offsetCommitTimer in `onJoinPrepare`. 2. The new parameter `max.offset.auto.commit.in.rebalance.ms` I mentioned is **only used** in `onJoinPrepare` instead of `time.timer(rebalanceTimeoutMs)`. In fact, the `consumer.commitSync` will not call `maybeAutoCommitOffsetsSync(Timer timer)` .If other callers call `commitOffsetsSync`, the timer can keep the status quo. ``` protected void onJoinPrepare(int generation, String memberId) { // commit offsets prior to rebalance if auto-commit enabled maybeAutoCommitOffsetsSync(time.timer(rebalanceTimeoutMs)); ..... } ``` 3. We need to consider the method `public void commitSync (Duration timeout)` in `kafkaConsumer`, whether `timeout` can continue to be set by customers. Customers may not know what impact `timeout` will bring. Customers simply thought that they just submitted the offset synchronously within the set time range, ignoring that a large number of requests would be initiated to the broker, resulting in a sharp increase in broker side cpu and even the collapse of the entire cluster. Precondition:There are many consuemr clients on the line calling `commitSync((Duration.ofMillis(60000)))` at the same time. I set 60s here, and it may be set for a longer time. If the conditions described in jira appear, some topics that are being subscribed will be deleted. Results: (1). All consumers will be blocked in commitSync again, maybe the customer will go to poll after commitSync, so the customer's consumption progress is suspended. (2). The consumer has been cyclically retrying to submit the offset, and the retry interval is very short: `timer.sleep(rebalanceConfig.retryBackoffMs)`, the broker-side cpu will increase significantly. 4. Do we have to choose a synchronous method for submitting the offset in `onJoinPrepare`? Can we provide parameters for the customer to choose synchronous or asynchronous? It is just a suggestion. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org