[jira] [Updated] (SPARK-38661) [TESTS] Replace 'abc & Symbol("abc") symbols with $"abc" in tests

2022-03-25 Thread Martin Tzvetanov Grigorov (Jira)


 [ 
https://issues.apache.org/jira/browse/SPARK-38661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Tzvetanov Grigorov updated SPARK-38661:
--
Description: 
This ticket is a follow up of SPARK-38351.

 

When building with Scala 2.13 many test classes produce warnings like:
{code:java}
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("d") instead
[warn]   'd.cast("string"),
[warn]   ^
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("e") instead
[warn]   'e.cast("string")).collect())
 {code}
For easier migration to Scala 3.x later it would be good to fix this warnings!

 

Also as suggested by [https://github.com/HeartSaVioR] it would be good to use 
Spark's $"abc" syntax for columns.

  was:
When building with Scala 2.13 many test classes produce warnings like:
{code:java}
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("d") instead
[warn]   'd.cast("string"),
[warn]   ^
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("e") instead
[warn]   'e.cast("string")).collect())
 {code}
For easier migration to Scala 3.x later it would be good to fix this warnings!


> [TESTS] Replace 'abc & Symbol("abc") symbols with $"abc" in tests
> -
>
> Key: SPARK-38661
> URL: https://issues.apache.org/jira/browse/SPARK-38661
> Project: Spark
>  Issue Type: Improvement
>  Components: Tests
>Affects Versions: 3.2.1
>Reporter: Martin Tzvetanov Grigorov
>Assignee: Martin Tzvetanov Grigorov
>Priority: Minor
> Fix For: 3.3.0
>
>
> This ticket is a follow up of SPARK-38351.
>  
> When building with Scala 2.13 many test classes produce warnings like:
> {code:java}
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("d") instead
> [warn]   'd.cast("string"),
> [warn]   ^
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("e") instead
> [warn]   'e.cast("string")).collect())
>  {code}
> For easier migration to Scala 3.x later it would be good to fix this warnings!
>  
> Also as suggested by [https://github.com/HeartSaVioR] it would be good to use 
> Spark's $"abc" syntax for columns.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38661) [TESTS] Replace 'abc & Symbol("abc") symbols with $"abc" in tests

2022-03-25 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38661:
-

 Summary: [TESTS] Replace 'abc & Symbol("abc") symbols with $"abc" 
in tests
 Key: SPARK-38661
 URL: https://issues.apache.org/jira/browse/SPARK-38661
 Project: Spark
  Issue Type: Improvement
  Components: Tests
Affects Versions: 3.2.1
Reporter: Martin Tzvetanov Grigorov
Assignee: Martin Tzvetanov Grigorov
 Fix For: 3.3.0


When building with Scala 2.13 many test classes produce warnings like:
{code:java}
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("d") instead
[warn]   'd.cast("string"),
[warn]   ^
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("e") instead
[warn]   'e.cast("string")).collect())
 {code}
For easier migration to Scala 3.x later it would be good to fix this warnings!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-38351) [TESTS] Replace 'abc symbols with Symbol("abc") in tests

2022-03-03 Thread Martin Tzvetanov Grigorov (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-38351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500812#comment-17500812
 ] 

Martin Tzvetanov Grigorov commented on SPARK-38351:
---

Yes, there are more !

Related to [https://github.com/apache/spark/pull/35560#issuecomment-1044535505] 
- I wanted the PR to be not too big for two reasons:

1) the review will be too long and tedious;

2) bigger chance for merge conflicts. I expect the more active developers in 
[SQL] to have some issues now.

> [TESTS] Replace 'abc symbols with Symbol("abc") in tests
> 
>
> Key: SPARK-38351
> URL: https://issues.apache.org/jira/browse/SPARK-38351
> Project: Spark
>  Issue Type: Improvement
>  Components: Tests
>Affects Versions: 3.2.1
>Reporter: Martin Tzvetanov Grigorov
>Assignee: Martin Tzvetanov Grigorov
>Priority: Minor
> Fix For: 3.3.0
>
>
> When building with Scala 2.13 many test classes produce warnings like:
> {code:java}
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("d") instead
> [warn]   'd.cast("string"),
> [warn]   ^
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("e") instead
> [warn]   'e.cast("string")).collect())
>  {code}
> For easier migration to Scala 3.x later it would be good to fix this warnings!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-38351) [TESTS] Replace 'abc symbols with Symbol("abc") in tests

2022-03-03 Thread Martin Tzvetanov Grigorov (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-38351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17500802#comment-17500802
 ] 

Martin Tzvetanov Grigorov commented on SPARK-38351:
---

[~srowen]  Should I open a new Jira ticket for the next batch of Symbol() 
changes ? In my previous comment I've asked to keep this ticket open but maybe 
Spark project prefers separate tickets for each PR ?

> [TESTS] Replace 'abc symbols with Symbol("abc") in tests
> 
>
> Key: SPARK-38351
> URL: https://issues.apache.org/jira/browse/SPARK-38351
> Project: Spark
>  Issue Type: Improvement
>  Components: Tests
>Affects Versions: 3.2.1
>Reporter: Martin Tzvetanov Grigorov
>Assignee: Martin Tzvetanov Grigorov
>Priority: Minor
> Fix For: 3.3.0
>
>
> When building with Scala 2.13 many test classes produce warnings like:
> {code:java}
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("d") instead
> [warn]   'd.cast("string"),
> [warn]   ^
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("e") instead
> [warn]   'e.cast("string")).collect())
>  {code}
> For easier migration to Scala 3.x later it would be good to fix this warnings!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38392) [K8S] Improve the names used for K8S namespace and driver pod in integration tests

2022-03-02 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38392:
-

 Summary: [K8S] Improve the names used for K8S namespace and driver 
pod in integration tests
 Key: SPARK-38392
 URL: https://issues.apache.org/jira/browse/SPARK-38392
 Project: Spark
  Issue Type: Improvement
  Components: Kubernetes
Affects Versions: 3.3.0
Reporter: Martin Tzvetanov Grigorov


Currently when there is no configured K8S namespace by the user the Kubernetes 
IT tests use UUID.toString() without the '-'es:
{code:java}
val namespace = 
namespaceOption.getOrElse(UUID.randomUUID().toString.replaceAll("-", "")) {code}
 

Proposal: prefix the temporary namespace with "spark-" or "spark-test-".

 

The name of the driver pod is:
{code:java}
driverPodName = "spark-test-app-" + UUID.randomUUID().toString.replaceAll("-", 
"") {code}
i.e. it does not mention that it is the driver.

For non-test it uses names like `spark-on-k8s-app-f6bfc57f4a938abe-driver`, 
i.e. the value of "–name" config + hash + "-driver".

In both IT tests and non-test the executor pods always mention "-exec-", so 
they are clear.

 

Proposal: unify non-test and IT test name and use "spark-test-app-"+ hash + 
"-driver" for the IT tests.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Updated] (SPARK-38362) Move eclipse.m2e Maven plugin config in its own profile

2022-03-01 Thread Martin Tzvetanov Grigorov (Jira)


 [ 
https://issues.apache.org/jira/browse/SPARK-38362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Martin Tzvetanov Grigorov updated SPARK-38362:
--
Description: 
Today I had a weird issue with org.eclipse.m2e:lifecycle-mapping Maven 
fake-plugin:
{code:java}
WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no 
dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 was not found in 
https://maven-central.storage-download.googleapis.com/maven2/ during a previous 
attempt. This failure was cached in the local repository and resolution is not 
reattempted until the update interval of gcs-maven-central-mirror has elapsed 
or updates are forced {code}
 

It was weird because I didn't do any changes to my setup since yesterday when 
the maven build was working fine.

{*}T{*}he actual problem was that ./dev/make-distribution was failing to read 
the version from pom.xml. The warnings above was the only thing printed by "mvn 
help:evaluate -Dexpression=project.version" so I thought it was related and 
spent time investigating it. There is no need other developers to waste time on 
Eclipse M2E warnings!

 

org.eclipse.m2e:lifecycle-mapping is a hack that is used by Eclipse to map 
Maven plugins' lifecycle with Eclipse lifecycle. It does not affect plain Maven 
usage on the command line! There is no Maven artifact at 
[https://repo.maven.apache.org/maven2/org/eclipse/m2e] !

 

As explained at [https://stackoverflow.com/a/23707050/497381] the best way to 
setup Maven+m2e is by using a custom Maven profile that is auto-activated only 
by Eclipse when M2E plugin is being used:
{code:java}

  only-eclipse
  

  m2e.version

  
  

  

  org.eclipse.m2e
  lifecycle-mapping
  1.0.0
  
...
  

  

  
 {code}

  was:
Today I had a weird issue with org.eclipse.m2e:lifecycle-mapping Maven 
fake-plugin:
{code:java}
WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no 
dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 was not found in 
https://maven-central.storage-download.googleapis.com/maven2/ during a previous 
attempt. This failure was cached in the local repository and resolution is not 
reattempted until the update interval of gcs-maven-central-mirror has elapsed 
or updates are forced {code}
 

It was weird because I didn't do any changes to my setup since yesterday when 
the maven build was working fine.

 

org.eclipse.m2e:lifecycle-mapping is a hack that is used by Eclipse to map 
Maven plugins' lifecycle with Eclipse lifecycle. It does not affect plain Maven 
usage on the command line! There is no Maven artifact at 
[https://repo.maven.apache.org/maven2/org/eclipse/m2e] !

 

As explained at [https://stackoverflow.com/a/23707050/497381] the best way to 
setup Maven+m2e is by using a custom Maven profile that is auto-activated only 
by Eclipse when M2E plugin is being used:
{code:java}

  only-eclipse
  

  m2e.version

  
  

  

  org.eclipse.m2e
  lifecycle-mapping
  1.0.0
  
...
  

  

  
 {code}


> Move eclipse.m2e Maven plugin config in its own profile
> ---
>
> Key: SPARK-38362
> URL: https://issues.apache.org/jira/browse/SPARK-38362
> Project: Spark
>  Issue Type: Improvement
>  Components: Build
>Affects Versions: 3.3.0
>Reporter: Martin Tzvetanov Grigorov
>Priority: Minor
>
> Today I had a weird issue with org.eclipse.m2e:lifecycle-mapping Maven 
> fake-plugin:
> {code:java}
> WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, 
> no dependency information available
> [WARNING] Failed to retrieve plugin descriptor for 
> org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
> org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not 
> be resolved: org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 was not found in 
> https://maven-central.storage-download.googleapis.com/maven2/ during a 
> previous attempt. This failure was cached in the local repository and 
> resolution is not reattempted until the update interval of 
> gcs-maven-central-mirror has elapsed or updates are forced {code}
>  
> It was weird because I didn't do any changes to my setup since yesterday when 
> the maven build 

[jira] [Created] (SPARK-38362) Move eclipse.m2e Maven plugin config in its own profile

2022-03-01 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38362:
-

 Summary: Move eclipse.m2e Maven plugin config in its own profile
 Key: SPARK-38362
 URL: https://issues.apache.org/jira/browse/SPARK-38362
 Project: Spark
  Issue Type: Improvement
  Components: Build
Affects Versions: 3.3.0
Reporter: Martin Tzvetanov Grigorov


Today I had a weird issue with org.eclipse.m2e:lifecycle-mapping Maven 
fake-plugin:
{code:java}
WARNING] The POM for org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 is missing, no 
dependency information available
[WARNING] Failed to retrieve plugin descriptor for 
org.eclipse.m2e:lifecycle-mapping:1.0.0: Plugin 
org.eclipse.m2e:lifecycle-mapping:1.0.0 or one of its dependencies could not be 
resolved: org.eclipse.m2e:lifecycle-mapping:jar:1.0.0 was not found in 
https://maven-central.storage-download.googleapis.com/maven2/ during a previous 
attempt. This failure was cached in the local repository and resolution is not 
reattempted until the update interval of gcs-maven-central-mirror has elapsed 
or updates are forced {code}
 

It was weird because I didn't do any changes to my setup since yesterday when 
the maven build was working fine.

 

org.eclipse.m2e:lifecycle-mapping is a hack that is used by Eclipse to map 
Maven plugins' lifecycle with Eclipse lifecycle. It does not affect plain Maven 
usage on the command line! There is no Maven artifact at 
[https://repo.maven.apache.org/maven2/org/eclipse/m2e] !

 

As explained at [https://stackoverflow.com/a/23707050/497381] the best way to 
setup Maven+m2e is by using a custom Maven profile that is auto-activated only 
by Eclipse when M2E plugin is being used:
{code:java}

  only-eclipse
  

  m2e.version

  
  

  

  org.eclipse.m2e
  lifecycle-mapping
  1.0.0
  
...
  

  

  
 {code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Commented] (SPARK-38351) [TESTS] Replace 'abc symbols with Symbol("abc") in tests

2022-02-28 Thread Martin Tzvetanov Grigorov (Jira)


[ 
https://issues.apache.org/jira/browse/SPARK-38351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17499373#comment-17499373
 ] 

Martin Tzvetanov Grigorov commented on SPARK-38351:
---

Please do not close this ticket when 
[https://github.com/apache/spark/pull/35560] is merged!

There will be few more PRs after it!

The task will be split in several PRs so that it is easier to review them.

> [TESTS] Replace 'abc symbols with Symbol("abc") in tests
> 
>
> Key: SPARK-38351
> URL: https://issues.apache.org/jira/browse/SPARK-38351
> Project: Spark
>  Issue Type: Improvement
>  Components: Tests
>Affects Versions: 3.2.1
>Reporter: Martin Tzvetanov Grigorov
>Priority: Minor
>
> When building with Scala 2.13 many test classes produce warnings like:
> {code:java}
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("d") instead
> [warn]   'd.cast("string"),
> [warn]   ^
> [warn] 
> /home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
>  [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; 
> use Symbol("e") instead
> [warn]   'e.cast("string")).collect())
>  {code}
> For easier migration to Scala 3.x later it would be good to fix this warnings!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38351) [TESTS] Replace 'abc symbols with Symbol("abc") in tests

2022-02-28 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38351:
-

 Summary: [TESTS] Replace 'abc symbols with Symbol("abc") in tests
 Key: SPARK-38351
 URL: https://issues.apache.org/jira/browse/SPARK-38351
 Project: Spark
  Issue Type: Improvement
  Components: Tests
Affects Versions: 3.2.1
Reporter: Martin Tzvetanov Grigorov


When building with Scala 2.13 many test classes produce warnings like:
{code:java}
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:562:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("d") instead
[warn]   'd.cast("string"),
[warn]   ^
[warn] 
/home/runner/work/spark/spark/sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala:563:11:
 [deprecation @  | origin= | version=2.13.0] symbol literal is deprecated; use 
Symbol("e") instead
[warn]   'e.cast("string")).collect())
 {code}
For easier migration to Scala 3.x later it would be good to fix this warnings!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38242) Sort the SparkSubmit debug output

2022-02-17 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38242:
-

 Summary: Sort the SparkSubmit debug output 
 Key: SPARK-38242
 URL: https://issues.apache.org/jira/browse/SPARK-38242
 Project: Spark
  Issue Type: Wish
  Components: Spark Submit
Affects Versions: 3.2.1
Reporter: Martin Tzvetanov Grigorov


When '-verbose' is passed to SparkSubmit it prints some useful debug 
information: Main class, Argument and Spark config.

I find it a bit hard to find information there because the arguments/configs 
are shuffled. I suggest to sort those before printing them.

 
{code:java}
 Main class:
org.apache.spark.deploy.k8s.submit.KubernetesClientApplication
Arguments:
--main-class
--primary-java-resource
local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar
org.apache.spark.examples.SparkPi
Spark config:
(spark.app.name,spark-on-k8s-app)
(spark.app.submitTime,1645106476125)
(spark.driver.cores,1)
(spark.driver.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
(spark.driver.memory,2048m)
(spark.dynamicAllocation.enabled,true)
(spark.dynamicAllocation.shuffleTracking.enabled,true)
(spark.executor.cores,2)
(spark.executor.extraJavaOptions,-Dio.netty.tryReflectionSetAccessible=true)
(spark.executor.instances,3)
(spark.executor.memory,2048m)
(spark.jars,local:///opt/spark/examples/jars/spark-examples_2.13-3.3.0-SNAPSHOT.jar)
(spark.kubernetes.allocation.batch.delay,1)
(spark.kubernetes.allocation.batch.size,3)
(spark.kubernetes.authenticate.driver.serviceAccountName,spark-account-name)
(spark.kubernetes.driver.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
(spark.kubernetes.executor.container.image,spark/spark:3.3.0-SNAPSHOT-scala_2.13-11-jre-slim)
(spark.kubernetes.namespace,spark-on-k8s)
(spark.master,k8s://https://192.168.49.2:8443)
(spark.network.timeout,300)
(spark.submit.deployMode,cluster)
(spark.submit.pyFiles,)
Classpath elements:{code}
 

The "Parsed arguments:" order is hardcoded at 
org.apache.spark.deploy.SparkSubmitArguments#toString, so they are still 
shuffled.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38241) Close KubernetesClient in K8S integrations tests

2022-02-17 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38241:
-

 Summary: Close KubernetesClient in K8S integrations tests
 Key: SPARK-38241
 URL: https://issues.apache.org/jira/browse/SPARK-38241
 Project: Spark
  Issue Type: Task
  Components: Kubernetes, Tests
Affects Versions: 3.2.1
Reporter: Martin Tzvetanov Grigorov


The implementations of 
org.apache.spark.deploy.k8s.integrationtest.backend.IntegrationTestBackend 
should close their KubernetesClient instance in #cleanUp() method.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org



[jira] [Created] (SPARK-38096) [BUILD] Update sbt to 1.6.2

2022-02-03 Thread Martin Tzvetanov Grigorov (Jira)
Martin Tzvetanov Grigorov created SPARK-38096:
-

 Summary: [BUILD] Update sbt to 1.6.2
 Key: SPARK-38096
 URL: https://issues.apache.org/jira/browse/SPARK-38096
 Project: Spark
  Issue Type: Task
  Components: Build
Affects Versions: 3.2.1
Reporter: Martin Tzvetanov Grigorov


sbt 1.6.2 has been released few days ago - https://eed3si9n.com/sbt-1.6.2



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org