[ https://issues.apache.org/jira/browse/SPARK-6629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388223#comment-14388223 ]
Apache Spark commented on SPARK-6629: ------------------------------------- User 'JoshRosen' has created a pull request for this issue: https://github.com/apache/spark/pull/5288 > cancelJobGroup() may not work for jobs whose job groups are inherited from > parent threads > ----------------------------------------------------------------------------------------- > > Key: SPARK-6629 > URL: https://issues.apache.org/jira/browse/SPARK-6629 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 1.0.2, 1.1.2, 1.2.2, 1.3.0 > Reporter: Josh Rosen > Assignee: Josh Rosen > Priority: Minor > > When a job is submitted with a job group and that job group is inherited from > a parent thread, there are multiple bugs that may prevent this job from being > cancelable via SparkContext.cancelJobGroup(): > - When filtering jobs based on their job group properties, DAGScheduler calls > get() instead of getProperty(), which does not respect inheritance, so it > will skip over jobs whose groups were inherited. > - Properties objects are mutable, but we do not make defensive copies / > snapshots, so modifications of the parent thread's job group will cause > running jobs' groups to change. > Both of these issues are easy to fix: use `getProperty()` and perform > defensive copying. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org