[
https://issues.apache.org/jira/browse/SPARK-38242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen reassigned SPARK-38242:
------------------------------------
Assignee: Martin Tzvetanov Grigorov
> Sort the SparkSubmit debug output
> ----------------------------------
>
> Key: SPARK-38242
> URL: https://issues.apache.org/jira/browse/SPARK-38242
> Project: Spark
> Issue Type: Wish
> Components: Spark Submit
> Affects Versions: 3.2.1
> Reporter: Martin Tzvetanov Grigorov
> Assignee: Martin Tzvetanov Grigorov
> Priority: Minor
>
> When '-verbose' is passed to SparkSubmit it prints some useful debug
> information: Main class, Argument and Spark config.
> I find it a bit hard to find information there because the arguments/configs
> are shuffled. I suggest to sort those before printing them.
>
> {code:java}
> Main class:
> org.apache.spark.deploy.k8s.submit.KubernetesClientApplication
> Arguments:
> --main-class
> --primary-java-resource
> local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar
> org.apache.spark.examples.SparkPi
> Spark config:
> (spark.app.name,spark-on-k8s-app)
> (spark.app.submitTime,1645106476125)
> (spark.driver.cores,1)
> (spark.driver.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
> (spark.driver.memory,2048m)
> (spark.dynamicAllocation.enabled,true)
> (spark.dynamicAllocation.shuffleTracking.enabled,true)
> (spark.executor.cores,2)
> (spark.executor.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
> (spark.executor.instances,3)
> (spark.executor.memory,2048m)
> (spark.jars,local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar)
> (spark.kubernetes.allocation.batch.delay,1)
> (spark.kubernetes.allocation.batch.size,3)
> (spark.kubernetes.authenticate.driver.serviceAccountName,spark-account-name)
> (spark.kubernetes.driver.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
> (spark.kubernetes.executor.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
> (spark.kubernetes.namespace,spark-on-k8s)
> (spark.master,k8s://https://192.168.49.2:8443)
> (spark.network.timeout,300)
> (spark.submit.deployMode,cluster)
> (spark.submit.pyFiles,)
> Classpath elements:{code}
>
> The "Parsed arguments:" order is hardcoded at
> org.apache.spark.deploy.SparkSubmitArguments#toString, so they are still
> shuffled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
