[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598393#comment-16598393 ]
Onur Satici edited comment on SPARK-24434 at 8/31/18 8:00 AM: -------------------------------------------------------------- Hello [~felixcheung], sorry I think the miscommunication here was because of the discrepancy between this Jira and k8s-sig-big-data weekly meeting notes. On [15 Aug|https://docs.google.com/document/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit#heading=h.d1p209nfiamv] it was discussed that as [~skonto] was out, and was not actively working on this PR at that moment, [~yifeih] and I can take over and start working on this. I think we should have reflected that decision in this Jira after the meeting to clear our intentions. We needed this change urgently as we had a couple of PR's adding new spark configuration options to customize Spark pods, and they were all blocked by this. was (Author: onursatici): Hello [~felixcheung], sorry I think the miscommunication here was because of the discrepancy between this Jira and k8s-sig-big-data weekly meeting notes. On [link 15 Aug|https://docs.google.com/document/d/1pnF38NF6N5eM8DlK088XUW85Vms4V2uTsGZvSp8MNIA/edit#heading=h.d1p209nfiamv] it was discussed that as [~skonto] was out, and was not actively working on this PR at that moment, [~yifeih] and I can take over and start working on this. I think we should have reflected that decision in this Jira after the meeting to clear our intentions. We needed this change urgently as we had a couple of PR's adding new spark configuration options to customize Spark pods, and they were all blocked by this. > Support user-specified driver and executor pod templates > -------------------------------------------------------- > > Key: SPARK-24434 > URL: https://issues.apache.org/jira/browse/SPARK-24434 > Project: Spark > Issue Type: New Feature > Components: Kubernetes > Affects Versions: 2.4.0 > Reporter: Yinan Li > Priority: Major > > With more requests for customizing the driver and executor pods coming, the > current approach of adding new Spark configuration options has some serious > drawbacks: 1) it means more Kubernetes specific configuration options to > maintain, and 2) it widens the gap between the declarative model used by > Kubernetes and the configuration model used by Spark. We should start > designing a solution that allows users to specify pod templates as central > places for all customization needs for the driver and executor pods. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org