dongjoon-hyun commented on a change in pull request #24630: [SPARK-27754][K8S]
Introduce config for driver request cores
URL: https://github.com/apache/spark/pull/24630#discussion_r284968318
##########
File path:
resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicDriverFeatureStepSuite.scala
##########
@@ -117,6 +117,33 @@ class BasicDriverFeatureStepSuite extends SparkFunSuite {
assert(featureStep.getAdditionalPodSystemProperties() ===
expectedSparkConf)
}
+ test("Check driver pod respects kubernetes driver request cores") {
+ val sparkConf = new SparkConf()
+ .set(KUBERNETES_DRIVER_POD_NAME, "spark-driver-pod")
+ .set(CONTAINER_IMAGE, "spark-driver:latest")
+
+ val basePod = SparkPod.initialPod()
+ val requests1 = new
BasicDriverFeatureStep(KubernetesTestConf.createDriverConf(sparkConf))
+ .configurePod(basePod)
+ .container.getResources
+ .getRequests.asScala
+ assert(requests1("cpu").getAmount === "1")
+
+ sparkConf.set(KUBERNETES_DRIVER_REQUEST_CORES, "0.1")
+ val requests2 = new
BasicDriverFeatureStep(KubernetesTestConf.createDriverConf(sparkConf))
+ .configurePod(basePod)
+ .container.getResources
+ .getRequests.asScala
+ assert(requests2("cpu").getAmount === "0.1")
+
+ sparkConf.set(KUBERNETES_DRIVER_REQUEST_CORES, "100m")
+ val requests3 = new
BasicDriverFeatureStep(KubernetesTestConf.createDriverConf(sparkConf))
+ .configurePod(basePod)
+ .container.getResources
+ .getRequests.asScala
+ assert(requests3("cpu").getAmount === "100m")
Review comment:
If you don't mind, could you avoid repetitions like the following?
```scala
Seq("0.1", "100m").foreach { value =>
sparkConf.set(KUBERNETES_DRIVER_REQUEST_CORES, value)
val requests = new
BasicDriverFeatureStep(KubernetesTestConf.createDriverConf(sparkConf))
.configurePod(basePod)
.container.getResources
.getRequests.asScala
assert(requests("cpu").getAmount === value)
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
