[
https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787090#comment-16787090
]
shane knapp commented on SPARK-26742:
-------------------------------------
integration tests passed for [https://github.com/apache/spark/pull/24002] !
{noformat}
[INFO] --- scalatest-maven-plugin:1.0:test (integration-test) @
spark-kubernetes-integration-tests_2.12 ---
Discovery starting.
Discovery completed in 173 milliseconds.
Run starting. Expected test count is: 15
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark with Python2 to test a pyfiles example
- Run PySpark with Python3 to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
Run completed in 7 minutes, 27 seconds.
Total number of tests run: 15
Suites: completed 2, aborted 0
Tests: succeeded 15, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 3.425 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 3.172 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 2.374 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 3.852 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 2.264 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 2.428 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 2.996 s]
[INFO] Spark Project Core ................................. SUCCESS [ 20.405 s]
[INFO] Spark Project Kubernetes Integration Tests ......... SUCCESS [08:37 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 09:18 min
[INFO] Finished at: 2019-03-07T10:34:37-08:00
[INFO]
------------------------------------------------------------------------{noformat}
i'll send notice to the dev@ list that i'll be updating the workers to minikube
v.0.34.1 and k8s 1.13.3 this afternoon.
i'll temporarily pause the PRB triggers for spark during the upgrade. this
should realistically take less than 30min as i've got everything staged.
when that's all done, i'll turn on the PRB builds again and trigger these two
PRs. when they pass, i'll mark these stuff resolved and we can merge.
sound good?
> Bump Kubernetes Client Version to 4.1.2
> ---------------------------------------
>
> Key: SPARK-26742
> URL: https://issues.apache.org/jira/browse/SPARK-26742
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Kubernetes
> Affects Versions: 2.4.0, 3.0.0
> Reporter: Steve Davids
> Priority: Major
> Labels: easyfix
> Fix For: 3.0.0
>
>
> Spark 2.x is using Kubernetes Client 3.x which is pretty old, the master
> branch has 4.0, the client should be upgraded to 4.1.1 to have the broadest
> Kubernetes compatibility support for newer clusters:
> https://github.com/fabric8io/kubernetes-client#compatibility-matrix
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
