[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's ann
hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#discussion_r263679533 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria set("spark.master", master) } - /** Set a name for your application. Shown in the Spark web UI. */ + /** Set a name for your application. Shown in the Spark web UI. +* For spark on kubernetes,Unify the three variables' name : Review comment: Thank you for your reply. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's ann
hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#discussion_r263628017 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria set("spark.master", master) } - /** Set a name for your application. Shown in the Spark web UI. */ + /** Set a name for your application. Shown in the Spark web UI. +* For spark on kubernetes,Unify the three variables' name : Review comment: /** Set a name for your application. Shown in the Spark web UI. * For spark on kubernetes,Unify the three variables' name : * pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations * */ It uses the value of spark.app.name to name the pod name for spark on k8s when submitting tasks by spark-submit . Meanwhile KubernetesClientApplication calls start function : `start(args: Array[String], conf: SparkConf)` in conf parameter, the value of spark.app.name equals the value of "--name". But the value of spark.app.name is not coincident. `val conf = new SparkConf().**setAppName("JavaNetworkWordCount")` `_ui = if (conf.getBoolean("spark.ui.enabled", true)) { Some(SparkUI.create(Some(this), _statusStore, _conf, _env.securityManager, appName, "", startTime)) } else { // For tests, do not enable the UI None }` Create an UI by calling SparkUI's create function which contains _conf parameter. The value of spark.app.name is setting by calling setAppName method. KubernetesClientApplication.start and SparkUI.create ,both functions have SparkConf parameter. They have the possibility of inconsistency for the value of spark.app.name in SparkConf. Refer to the pictures and content in the previous comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's ann
hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#discussion_r263628017 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria set("spark.master", master) } - /** Set a name for your application. Shown in the Spark web UI. */ + /** Set a name for your application. Shown in the Spark web UI. +* For spark on kubernetes,Unify the three variables' name : Review comment: /** Set a name for your application. Shown in the Spark web UI. * For spark on kubernetes,Unify the three variables' name : * pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations * */ It uses the value of "-- name" to name the pod name for spark on k8s when submitting tasks by spark-submit . Meanwhile KubernetesClientApplication calls start function : `start(args: Array[String], conf: SparkConf)` in conf parameter, the value of spark.app.name equals the value of "--name". But the value of spark.app.name is not coincident. `val conf = new SparkConf().**setAppName("JavaNetworkWordCount")` `_ui = if (conf.getBoolean("spark.ui.enabled", true)) { Some(SparkUI.create(Some(this), _statusStore, _conf, _env.securityManager, appName, "", startTime)) } else { // For tests, do not enable the UI None }` Create an UI by calling SparkUI's create function which contains _conf parameter. The value of spark.app.name is setting by calling setAppName method. KubernetesClientApplication.start and SparkUI.create ,both functions have SparkConf parameter. They have the possibility of inconsistency for the value of spar.app.name in SparkConf. Refer to the pictures and content in the previous comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's ann
hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#discussion_r263628017 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria set("spark.master", master) } - /** Set a name for your application. Shown in the Spark web UI. */ + /** Set a name for your application. Shown in the Spark web UI. +* For spark on kubernetes,Unify the three variables' name : Review comment: /** Set a name for your application. Shown in the Spark web UI. * For spark on kubernetes,Unify the three variables' name : * pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations * */ It uses the value of "-- name" to name the pod name for spark on k8s when submitting tasks by spark-submit . Meanwhile KubernetesClientApplication calls start function : `start(args: Array[String], conf: SparkConf)` in conf parameter, the value of spark.app.name equals the value of "--name". But the value of spark.app.name is not coincident. `val conf = new SparkConf().**setAppName("JavaNetworkWordCount")` `_ui = if (conf.getBoolean("spark.ui.enabled", true)) { Some(SparkUI.create(Some(this), _statusStore, _conf, _env.securityManager, appName, "", startTime)) } else { // For tests, do not enable the UI None }` Create an UI by calling SparkUI's create function which contains _conf parameter. The value of spark.app.name is setting by calling setAppName method. KubernetesClientApplication.start and SparkUI.create ,both functions have SparkConf parameter. They have the possibility of inconsistency for the value of spark.app.name in SparkConf. Refer to the pictures and content in the previous comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's ann
hehuiyuan commented on a change in pull request #24009: [k8s]Unify the three variables' name : pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations URL: https://github.com/apache/spark/pull/24009#discussion_r263628017 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -115,9 +115,13 @@ class SparkConf(loadDefaults: Boolean) extends Cloneable with Logging with Seria set("spark.master", master) } - /** Set a name for your application. Shown in the Spark web UI. */ + /** Set a name for your application. Shown in the Spark web UI. +* For spark on kubernetes,Unify the three variables' name : Review comment: /** Set a name for your application. Shown in the Spark web UI. * For spark on kubernetes,Unify the three variables' name : * pod name prefix in kubernetes / spark.app.name in spark ui / spark-app-name in pod's annotations * */ It uses the value of "-- name" to name the pod name for spark on k8s when submitting tasks by spark-submit . Meanwhile KubernetesClientApplication calls start function : `start(args: Array[String], conf: SparkConf)` in conf parameter, the value of spark.app.name equals the value of "--name". But the value of spark.app.name is not coincident. `val conf = new SparkConf().**setAppName("JavaNetworkWordCount")` `_ui = if (conf.getBoolean("spark.ui.enabled", true)) { Some(SparkUI.create(Some(this), _statusStore, _conf, _env.securityManager, appName, "", startTime)) } else { // For tests, do not enable the UI None }` Create an UI by calling SparkUI's create function which contains _conf parameter. The value of spark.app.name is setting by calling setAppName method. KubernetesClientApplication.start and SparkUI.create ,both functions have SparkConf parameter. They have the possibility of inconsistency. Refer to the pictures and content in the previous comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org