[
https://issues.apache.org/jira/browse/SPARK-28367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Somogyi updated SPARK-28367:
----------------------------------
Description:
Spark uses an old and deprecated API named poll(long) which never returns and
stays in live lock if metadata is not updated (for instance when broker
disappears at consumer creation).
was:
Spark uses an old and deprecated API named poll(long) which never returns and
stays in live lock if metadata is not updated (for instance when broker
disappears at consumer creation).
I've created a small standalone application to test it and the alternatives:
https://github.com/gaborgsomogyi/kafka-get-assignment
> Kafka connector infinite wait because metadata never updated
> ------------------------------------------------------------
>
> Key: SPARK-28367
> URL: https://issues.apache.org/jira/browse/SPARK-28367
> Project: Spark
> Issue Type: Bug
> Components: Structured Streaming
> Affects Versions: 2.1.3, 2.2.3, 2.3.3, 2.4.3, 3.0.0, 3.1.0
> Reporter: Gabor Somogyi
> Priority: Critical
>
> Spark uses an old and deprecated API named poll(long) which never returns and
> stays in live lock if metadata is not updated (for instance when broker
> disappears at consumer creation).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
