[ https://issues.apache.org/jira/browse/SPARK-19182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu updated SPARK-19182: --------------------------------- Summary: Optimize the lock in StreamingJobProgressListener to not block UI when generating Streaming jobs (was: Optimize the lock in StreamingJobProgressListener to not block when generating Streaming jobs) > Optimize the lock in StreamingJobProgressListener to not block UI when > generating Streaming jobs > ------------------------------------------------------------------------------------------------ > > Key: SPARK-19182 > URL: https://issues.apache.org/jira/browse/SPARK-19182 > Project: Spark > Issue Type: Improvement > Components: DStreams > Reporter: Shixiong Zhu > Priority: Minor > > When DStreamGraph is generating a job, it will hold a lock and block other > APIs. Because StreamingJobProgressListener (numInactiveReceivers, > streamName(streamId: Int), streamIds) needs to call DStreamGraph's methods to > access some information, the UI may hang if generating a job is very slow > (e.g., talking to the slow Kafka cluster to fetch metadata). > It's better to optimize the locks in DStreamGraph and > StreamingJobProgressListener to make the UI not block by job generation. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org