[ https://issues.apache.org/jira/browse/KAFKA-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik van Oosten updated KAFKA-14972: ------------------------------------ Description: KafkaConsumer contains a check that rejects nested invocations from different threads (method {{{}acquire{}}}). For users that use an async runtime, this is an almost impossible requirement. Examples of async runtimes that are affected are Kotlin co-routines (see KAFKA-7143) and Zio. It should be possible for a thread to pass on its capability to access the consumer to another thread. See KIP-944 for a proposal and was: KafkaConsumer contains a check that rejects nested invocations from different threads (method {{{}acquire{}}}). For users that use an async runtime, this is an almost impossible requirement. Examples of async runtimes that are affected are Kotlin co-routines (see KAFKA-7143) and Zio. We propose to replace the thread-id check with an access-id that is stored on a thread-local variable. Existing programs will not be affected. Developers that work in an async runtime can pick up the access-id and set it on the thread-local variable in a thread of their choosing. Every time a callback is invoked a new access-id is generated. When the callback completes, the previous access-id is restored. This proposal does not make it impossible to use the client incorrectly. However, we think it strikes a good balance between making correct usage from an async runtime possible while making incorrect usage difficult. Alternatives considered: # Configuration that switches off the check completely. > Make KafkaConsumer usable in async runtimes > ------------------------------------------- > > Key: KAFKA-14972 > URL: https://issues.apache.org/jira/browse/KAFKA-14972 > Project: Kafka > Issue Type: Wish > Components: consumer > Reporter: Erik van Oosten > Assignee: Erik van Oosten > Priority: Major > Labels: needs-kip > > KafkaConsumer contains a check that rejects nested invocations from different > threads (method {{{}acquire{}}}). For users that use an async runtime, this > is an almost impossible requirement. Examples of async runtimes that are > affected are Kotlin co-routines (see KAFKA-7143) and Zio. > It should be possible for a thread to pass on its capability to access the > consumer to another thread. See KIP-944 for a proposal and -- This message was sent by Atlassian Jira (v8.20.10#820010)