[jira] [Commented] (SPARK-11137) Make StreamingContext.stop() exception-safe
[ https://issues.apache.org/jira/browse/SPARK-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105524#comment-15105524 ] Apache Spark commented on SPARK-11137: -- User 'jayadevanmurali' has created a pull request for this issue: https://github.com/apache/spark/pull/10807 > Make StreamingContext.stop() exception-safe > --- > > Key: SPARK-11137 > URL: https://issues.apache.org/jira/browse/SPARK-11137 > Project: Spark > Issue Type: Bug > Components: Streaming >Affects Versions: 1.5.1 >Reporter: Felix Cheung >Priority: Minor > > In StreamingContext.stop(), when an exception is thrown the rest of the > stop/cleanup action is aborted. > Discussed in https://github.com/apache/spark/pull/9116, > srowen commented > Hm, this is getting unwieldy. There are several nested try blocks here. The > same argument goes for many of these methods -- if one fails should they not > continue trying? A more tidy solution would be to execute a series of () -> > Unit code blocks that perform some cleanup and make sure that they each fire > in succession, regardless of the others. The final one to remove the shutdown > hook could occur outside synchronization. > I realize we're expanding the scope of the change here, but is it maybe > worthwhile to go all the way here? > Really, something similar could be done for SparkContext and there's an > existing JIRA for it somewhere. > At least, I'd prefer to either narrowly fix the deadlock here, or fix all of > the finally-related issue separately and all at once. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-11137) Make StreamingContext.stop() exception-safe
[ https://issues.apache.org/jira/browse/SPARK-11137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15105523#comment-15105523 ] Jayadevan M commented on SPARK-11137: - [~thomastechs] working on the fix > Make StreamingContext.stop() exception-safe > --- > > Key: SPARK-11137 > URL: https://issues.apache.org/jira/browse/SPARK-11137 > Project: Spark > Issue Type: Bug > Components: Streaming >Affects Versions: 1.5.1 >Reporter: Felix Cheung >Priority: Minor > > In StreamingContext.stop(), when an exception is thrown the rest of the > stop/cleanup action is aborted. > Discussed in https://github.com/apache/spark/pull/9116, > srowen commented > Hm, this is getting unwieldy. There are several nested try blocks here. The > same argument goes for many of these methods -- if one fails should they not > continue trying? A more tidy solution would be to execute a series of () -> > Unit code blocks that perform some cleanup and make sure that they each fire > in succession, regardless of the others. The final one to remove the shutdown > hook could occur outside synchronization. > I realize we're expanding the scope of the change here, but is it maybe > worthwhile to go all the way here? > Really, something similar could be done for SparkContext and there's an > existing JIRA for it somewhere. > At least, I'd prefer to either narrowly fix the deadlock here, or fix all of > the finally-related issue separately and all at once. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
