[
https://issues.apache.org/jira/browse/SPARK-26742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16786047#comment-16786047
]
shane knapp commented on SPARK-26742:
-------------------------------------
works for me against k8s v1.11.7:
{noformat}
[INFO]
[INFO] --- scalatest-maven-plugin:1.0:test (integration-test) @
spark-kubernetes-integration-tests_2.11 ---
Discovery starting.
Discovery completed in 181 milliseconds.
Run starting. Expected test count is: 14
KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark with Python2 to test a pyfiles example
- Run PySpark with Python3 to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
Run completed in 7 minutes, 1 second.
Total number of tests run: 14
Suites: completed 2, aborted 0
Tests: succeeded 14, failed 0, canceled 0, ignored 0, pending 0
All tests passed.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM 2.4.2-SNAPSHOT ............ SUCCESS [ 3.010 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 2.829 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 2.144 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 3.455 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 1.902 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 1.977 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 4.040 s]
[INFO] Spark Project Core ................................. SUCCESS [ 24.034 s]
[INFO] Spark Project Kubernetes Integration Tests 2.4.2-SNAPSHOT SUCCESS [08:03
min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:46 min
[INFO] Finished at: 2019-03-06T11:15:46-08:00
[INFO] ------------------------------------------------------------------------
jenkins@ubuntu-testing:~/src/spark$ grep version run-int-tests.sh
minikube --vm-driver=kvm2 start --memory 6000 --cpus 8
--kubernetes-version=v1.11.7
{noformat}
> Bump Kubernetes Client Version to 4.1.2
> ---------------------------------------
>
> Key: SPARK-26742
> URL: https://issues.apache.org/jira/browse/SPARK-26742
> Project: Spark
> Issue Type: Dependency upgrade
> Components: Kubernetes
> Affects Versions: 2.4.0, 3.0.0
> Reporter: Steve Davids
> Priority: Major
> Labels: easyfix
> Fix For: 3.0.0
>
>
> Spark 2.x is using Kubernetes Client 3.x which is pretty old, the master
> branch has 4.0, the client should be upgraded to 4.1.1 to have the broadest
> Kubernetes compatibility support for newer clusters:
> https://github.com/fabric8io/kubernetes-client#compatibility-matrix
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
