[ https://issues.apache.org/jira/browse/SPARK-23285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16632511#comment-16632511 ]
Apache Spark commented on SPARK-23285: -------------------------------------- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/22585 > Allow spark.executor.cores to be fractional > ------------------------------------------- > > Key: SPARK-23285 > URL: https://issues.apache.org/jira/browse/SPARK-23285 > Project: Spark > Issue Type: Improvement > Components: Kubernetes, Scheduler, Spark Submit > Affects Versions: 2.4.0 > Reporter: Anirudh Ramanathan > Assignee: Yinan Li > Priority: Minor > Fix For: 2.4.0 > > > There is a strong check for an integral number of cores per executor in > [SparkSubmitArguments.scala#L270-L272|https://github.com/apache/spark/blob/3f4060c340d6bac412e8819c4388ccba226efcf3/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala#L270-L272]. > Given we're reusing that property in K8s, does it make sense to relax it? > > K8s treats CPU as a "compressible resource" and can actually assign millicpus > to individual containers. Also to be noted - spark.driver.cores has no such > check in place. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org