[
https://issues.apache.org/jira/browse/SPARK-34738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17317418#comment-17317418
]
Shane Knapp edited comment on SPARK-34738 at 4/8/21, 6:40 PM:
--------------------------------------------------------------
{noformat}
Given path (/opt/spark/pv-tests/tmp5813668424419880732.txt) does not exist
{noformat}
this is supposed to be mounted in the minikube cluster, not on the bare metal.
> But sth is mounted to there:
{noformat}
Mounts:
/opt/spark/conf from spark-conf-volume-driver (rw)
/opt/spark/pv-tests from data (rw)
...
{noformat}
> But my guess this is not the one which connects the host path
> "PVC_TESTS_HOST_PATH" with the internal minikube mounted path (the one which
> goes further to the driver/executor). This is why the locally created file is
> missing.
it's properly mounting the local (bare metal) filesystem, as is able to create
the file. see:
https://issues.apache.org/jira/browse/SPARK-34738?focusedCommentId=17312548&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17312548
was (Author: shaneknapp):
{noformat}
Given path (/opt/spark/pv-tests/tmp5813668424419880732.txt) does not exist
{noformat}
this is supposed to be mounted in the minikube cluster, not on the bare metal.
> But sth is mounted to there:
{noformat}
Mounts:
/opt/spark/conf from spark-conf-volume-driver (rw)
/opt/spark/pv-tests from data (rw)
...
{noformat}
> But my guess this is not the one which connects the host path
> "PVC_TESTS_HOST_PATH" with the internal minikube mounted path (the one which
> goes further to the driver/executor). This is why the locally created file is
> missing.
it's properly mounting the local (bare metal) filesystem. see:
https://issues.apache.org/jira/browse/SPARK-34738?focusedCommentId=17312548&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17312548
> Upgrade Minikube and kubernetes cluster version on Jenkins
> ----------------------------------------------------------
>
> Key: SPARK-34738
> URL: https://issues.apache.org/jira/browse/SPARK-34738
> Project: Spark
> Issue Type: Task
> Components: jenkins, Kubernetes
> Affects Versions: 3.2.0
> Reporter: Attila Zsolt Piros
> Assignee: Shane Knapp
> Priority: Major
> Attachments: integration-tests.log
>
>
> [~shaneknapp] as we discussed [on the mailing
> list|http://apache-spark-developers-list.1001551.n3.nabble.com/minikube-and-kubernetes-cluster-versions-for-integration-testing-td30856.html]
> Minikube can be upgraded to the latest (v1.18.1) and kubernetes version
> should be v1.17.3 (`minikube config set kubernetes-version v1.17.3`).
> [Here|https://github.com/apache/spark/pull/31829] is my PR which uses a new
> method to configure the kubernetes client. Thanks in advance to use it for
> testing on the Jenkins after the Minikube version is updated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
