Thank you, Kirk! Your feedback is very much appreciated; I'd like to try to address them in the email and the document.
Here are my responses: 1. I removed that point because I think we should still try to maintain the current behavior to avoid regressions 2. I think I misworded it. I think I'm referring to the coordinator code. The example of the sync commit now relies on the polling thread to wait for the future to be completed instead of the background thread. I've made a note of that, and I'll make sure to clarify that section. 3. All issues with that Jira tag should be addressed by this refactoring. 4. Yes, it will be thread-safe. So locking mechanism will be added there. Noted. 5. Noted. 6. Thanks for bringing this up. I think I need to add a section on how to release these changes to AK. Thank you! P On Thu, Sep 15, 2022 at 2:38 PM Kirk True <k...@kirktrue.pro> wrote: > Hi Philip! > > Thanks for the write-up. > > On Tue, Sep 13, 2022, at 2:13 PM, Philip Nee wrote: > > Hi all, > > > > Here is the proposal to refactor the Kafka Consumer > > < > https://cwiki.apache.org/confluence/display/KAFKA/Proposal%3A+Consumer+Threading+Model+Refactor > >. > > The 1-pager is on the wiki, so please take a look at it. Also, this is a > > prerequisite for KIP-848 (the next gen rebalance protocol). > > I only have time for a quick read-through, but here are some initial > questions/comments: > > 1. The third bullet point in the "Public-Facing Changes" section says > that the "exception[s] thrown will be different." Can you provide some more > context on that? Will this affect user applications that attempt to handle > exceptions? > 2. Under "Scope" it mentions that the proposal is to "remove some > blocking methods, such as commitOffsetSync." Are you referring to the > Consumer.commitSync() method or something else? > 3. I like how the proposal will address the systemic issues with the > current consumer ( > https://issues.apache.org/jira/issues/?jql=labels%20%3D%20new-consumer-threading-should-fix). > Is there a specific set of those Jiras that will be fixed/resolved, or is > it 'best effort'? > 4. "A note on the *subscriptionState*: Its reference will be shared by > polling and background threads." Sharing this reference implies locking of > some sort, yes? > 5. Can you elaborate on this sentence: "We need to make sure the timing > of the 1. coordinator discovery and 2. joinGroup operations are being > done in the correct timing."? > 6. Does this new implementation for the consumer internals live alongside > the current implementation in the code base? How does a user opt-in to the > "new" implementation? > > > > > Cheers, > > P > > > > Thanks! > Kirk
