[jira] [Commented] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf
[ https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848522#comment-16848522 ] Apache Spark commented on SPARK-26192: -- User 'mwlon' has created a pull request for this issue: https://github.com/apache/spark/pull/24713 > MesosClusterScheduler reads options from dispatcher conf instead of > submission conf > --- > > Key: SPARK-26192 > URL: https://issues.apache.org/jira/browse/SPARK-26192 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.3.0, 2.3.1, 2.3.2, 2.4.0 >Reporter: Martin Loncaric >Assignee: Martin Loncaric >Priority: Minor > Fix For: 3.0.0 > > > There is at least one option accessed in MesosClusterScheduler that should > come from the submission's configuration instead of the dispatcher's: > spark.mesos.fetcherCache.enable > Coincidentally, the spark.mesos.fetcherCache.enable option was previously > misnamed, as referenced in the linked JIRA. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf
[ https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16848516#comment-16848516 ] Martin Loncaric commented on SPARK-26192: - [~dongjoon] I made a PR: https://github.com/apache/spark/pull/24713 > MesosClusterScheduler reads options from dispatcher conf instead of > submission conf > --- > > Key: SPARK-26192 > URL: https://issues.apache.org/jira/browse/SPARK-26192 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.3.0, 2.3.1, 2.3.2, 2.4.0 >Reporter: Martin Loncaric >Assignee: Martin Loncaric >Priority: Minor > Fix For: 3.0.0 > > > There is at least one option accessed in MesosClusterScheduler that should > come from the submission's configuration instead of the dispatcher's: > spark.mesos.fetcherCache.enable > Coincidentally, the spark.mesos.fetcherCache.enable option was previously > misnamed, as referenced in the linked JIRA. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf
[ https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16840867#comment-16840867 ] Dongjoon Hyun commented on SPARK-26192: --- Sorry for being late, [~mwlon]. Could you make a PR against `branch-2.4`? > MesosClusterScheduler reads options from dispatcher conf instead of > submission conf > --- > > Key: SPARK-26192 > URL: https://issues.apache.org/jira/browse/SPARK-26192 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.3.0, 2.3.1, 2.3.2, 2.4.0 >Reporter: Martin Loncaric >Assignee: Martin Loncaric >Priority: Minor > Fix For: 3.0.0 > > > There is at least one option accessed in MesosClusterScheduler that should > come from the submission's configuration instead of the dispatcher's: > spark.mesos.fetcherCache.enable > Coincidentally, the spark.mesos.fetcherCache.enable option was previously > misnamed, as referenced in the linked JIRA. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf
[ https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783810#comment-16783810 ] Martin Loncaric commented on SPARK-26192: - [~dongjoon] Thanks, I will pay more attention to those fields. However, I believe this is a bug. It violates behavior specified in the https://spark.apache.org/docs/latest/running-on-mesos.html#configuration. Can we merge into 2.4.1 as well? > MesosClusterScheduler reads options from dispatcher conf instead of > submission conf > --- > > Key: SPARK-26192 > URL: https://issues.apache.org/jira/browse/SPARK-26192 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.3.0, 2.3.1, 2.3.2, 2.4.0 >Reporter: Martin Loncaric >Assignee: Martin Loncaric >Priority: Minor > Fix For: 3.0.0 > > > There is at least one option accessed in MesosClusterScheduler that should > come from the submission's configuration instead of the dispatcher's: > spark.mesos.fetcherCache.enable > Coincidentally, the spark.mesos.fetcherCache.enable option was previously > misnamed, as referenced in the linked JIRA. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26192) MesosClusterScheduler reads options from dispatcher conf instead of submission conf
[ https://issues.apache.org/jira/browse/SPARK-26192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783745#comment-16783745 ] Dongjoon Hyun commented on SPARK-26192: --- [~mwlon]. Thank you for reporting and making a PR. However, please don't set 'Fix Versions`. `Fixed Version` and `Target Version` are used in the different way. Please refer the contribution guide. - https://spark.apache.org/contributing.html For me, this is a minor improvement for Spark 3.0. > MesosClusterScheduler reads options from dispatcher conf instead of > submission conf > --- > > Key: SPARK-26192 > URL: https://issues.apache.org/jira/browse/SPARK-26192 > Project: Spark > Issue Type: Improvement > Components: Mesos >Affects Versions: 2.3.0, 2.3.1, 2.3.2, 2.4.0 >Reporter: Martin Loncaric >Assignee: Martin Loncaric >Priority: Major > Fix For: 3.0.0 > > > There is at least one option accessed in MesosClusterScheduler that should > come from the submission's configuration instead of the dispatcher's: > spark.mesos.fetcherCache.enable > Coincidentally, the spark.mesos.fetcherCache.enable option was previously > misnamed, as referenced in the linked JIRA. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
