[jira] [Updated] (SPARK-20483) Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores

2017-04-26 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-20483: --- Description: if {{spark.cores.max = 10}} for example and {{spark.executor.cores = 4}}, 2

[jira] [Updated] (SPARK-20483) Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores

2017-04-26 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-20483: --- Description: if {{spark.cores.max = 10}} for example and {{spark.executor.cores = 4}}, 2

[jira] [Updated] (SPARK-20483) Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores

2017-04-26 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-20483: --- Description: if {{spark.cores.max = 10}} for example and {{spark.executor.cores = 4}}, 2

[jira] [Created] (SPARK-20483) Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores

2017-04-26 Thread Davis Shepherd (JIRA)
Davis Shepherd created SPARK-20483: -- Summary: Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores Key: SPARK-20483 URL:

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

2014-12-08 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14238221#comment-14238221 ] Davis Shepherd commented on SPARK-4759: --- The fix appears to resolve the issue in

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode with multiple cores

2014-12-07 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237264#comment-14237264 ] Davis Shepherd commented on SPARK-4759: --- It is possible to reproduce the issue with

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

2014-12-07 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237376#comment-14237376 ] Davis Shepherd commented on SPARK-4759: --- local[2] on the latest commit of branch 1.1

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

2014-12-07 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237383#comment-14237383 ] Davis Shepherd commented on SPARK-4759: --- Ok your version does reproduce the issue

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job in local mode

2014-12-07 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237393#comment-14237393 ] Davis Shepherd commented on SPARK-4759: --- I still cannot reproduce the issue in the

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job in local mode

2014-12-07 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14237393#comment-14237393 ] Davis Shepherd edited comment on SPARK-4759 at 12/8/14 3:15 AM:

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job.

2014-12-05 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235651#comment-14235651 ] Davis Shepherd commented on SPARK-4759: ---

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job.

2014-12-05 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235651#comment-14235651 ] Davis Shepherd edited comment on SPARK-4759 at 12/5/14 3:48 PM:

[jira] [Comment Edited] (SPARK-4759) Deadlock in complex spark job.

2014-12-05 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235651#comment-14235651 ] Davis Shepherd edited comment on SPARK-4759 at 12/5/14 3:52 PM:

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job.

2014-12-05 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Description: The attached test class runs two identical jobs that perform some iterative

[jira] [Commented] (SPARK-4759) Deadlock in complex spark job.

2014-12-05 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14235703#comment-14235703 ] Davis Shepherd commented on SPARK-4759: --- Fair enough. As far as I can work out, it

[jira] [Created] (SPARK-4759) Deadlock in pathological spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
Davis Shepherd created SPARK-4759: - Summary: Deadlock in pathological spark job. Key: SPARK-4759 URL: https://issues.apache.org/jira/browse/SPARK-4759 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4759) Deadlock in pathological spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Attachment: SparkBugReplicator.scala Deadlock in pathological spark job.

[jira] [Updated] (SPARK-4759) Deadlock in pathological spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Description: The attached test class runs two identical jobs that perform some iterative

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Summary: Deadlock in complex spark job. (was: Deadlock in pathological spark job.) Deadlock

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Attachment: (was: SparkBugReplicator.scala) Deadlock in complex spark job.

[jira] [Updated] (SPARK-4759) Deadlock in complex spark job.

2014-12-04 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-4759: -- Attachment: SparkBugReplicator.scala Deadlock in complex spark job.

[jira] [Commented] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

2014-10-16 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14173428#comment-14173428 ] Davis Shepherd commented on SPARK-3882: --- This is also a serious memory leak that

[jira] [Issue Comment Deleted] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

2014-10-16 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-3882: -- Comment: was deleted (was: This is also a serious memory leak that will cause long running

[jira] [Created] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

2014-10-09 Thread Davis Shepherd (JIRA)
Davis Shepherd created SPARK-3882: - Summary: JobProgressListener gets permanently out of sync with long running job Key: SPARK-3882 URL: https://issues.apache.org/jira/browse/SPARK-3882 Project:

[jira] [Updated] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

2014-10-09 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-3882: -- Attachment: Screen Shot 2014-10-03 at 12.50.59 PM.png Lots of orphaned jobs.

[jira] [Comment Edited] (SPARK-3882) JobProgressListener gets permanently out of sync with long running job

2014-10-09 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14165535#comment-14165535 ] Davis Shepherd edited comment on SPARK-3882 at 10/9/14 6:51 PM:

[jira] [Created] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.

2014-04-03 Thread Davis Shepherd (JIRA)
Davis Shepherd created SPARK-1411: - Summary: When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent. Key: SPARK-1411 URL: https://issues.apache.org/jira/browse/SPARK-1411

[jira] [Updated] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.

2014-04-03 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-1411: -- Attachment: Screen Shot 2014-04-03 at 5.35.00 PM.png Notice the large gap in stage ids and

[jira] [Updated] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.

2014-04-03 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd updated SPARK-1411: -- Priority: Major (was: Minor) When using spark.ui.retainedStages=n only the first n stages

[jira] [Closed] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.

2014-04-03 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davis Shepherd closed SPARK-1411. - Resolution: Duplicate When using spark.ui.retainedStages=n only the first n stages are kept,

[jira] [Commented] (SPARK-1411) When using spark.ui.retainedStages=n only the first n stages are kept, not the most recent.

2014-04-03 Thread Davis Shepherd (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13959664#comment-13959664 ] Davis Shepherd commented on SPARK-1411: --- I submitted a pull request to fix 1337 as